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Una Ricerca-Formazione, per essere tale, richiede: 

1. una esplicitazione chiara della finalità della ricerca in termini di crescita e sviluppo della professionalità degli 
insegnanti direttamente coinvolti e un'attenzione a documentare e analizzare le ricadute in termini di cambia- 
mento; 

2. la creazione di un gruppo di R-F di cui facciano parte ricercatore/i e insegnanti, nel quale vengano chiariti 


i diversi ruoli dei partecipanti e in cui vengano negoziati e chiariti obiettivi e oggetti, scelte valoriali e metodo- 
logiche della R-F; 
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Introduction 


by Giorgio Bolondi*, Ira Vannini*, Athanasios Gagatsis**, 
Laurent Jeannin**, Rob Velder**** 


The main purpose of project of research LLP Comenius Project 
“FAMT&L — Formative Assessment in Mathematics for Teaching and 
Learning” was to encourage the use of formative assessment (FA) in the 
process of teaching-learning; in particular within the teaching practices 
of mathematics teachers at the middle level of school (students aged 
from 11 to 16). 

The five member partners involved in the Project were the Alma 
Mater Studiorum Università di Bologna in Italy, the University of 
Applied Sciences and Arts of Southern Switzerland, the University of 
Cergy-Pontoise in France, the University of Cyprus, and the Netherlands” 
Hogeschool Inholland. 

For three years, researchers from the five countries worked together 
with associate school teachers, forming groups geared to promote teacher 
skills in the FA. The purpose of the group was therefore to make a change 
in the teachers’ beliefs and their assessment practices, both of the schools 
that participated in the project, and of other groups of teachers of broader 
contexts. The pathways have led the researchers to outline a specific 
training methodology in the FA field, with the development of specific 
techniques, tools and materials. 

The Project took place in the years 2013 to 2016. The realization of 
the pathways has followed some specific phases: analysis of contexts and 
teacher training needs; identification of techniques and tools; production 
of video training materials for training; design of pilot training paths and 
their implementation and monitoring. 


* Alma Mater Studiorum Università di Bologna. 
** University of Cyprus. 

*** Université de Cergy-Pontoise. 

*##k* Netherlands’ Hogeschool Inholland. 
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This book aims to describe the pathways of the Project and its main 
achievements: our purpose is to contribute to the international debate 
on issues on teachers’ professionalism in the school systems in Europe: 
specifically, the development of teachers’ professional skills in the field of 
assessment practices and, in particular, the use of the FA in classroom. The 
book is structured in ten chapters. 

The first chapter presents the scientific reasons for the adopted work 
methodology of “Teacher Professional Development Research” (Ricerca- 
Formazione, in Italian language). The principles of this approach and the 
resulting are discussed in the groups formed by university researchers and 
practicing teachers. Strengths and weaknesses are highlighted. 

The second chapter analyses the concept of FA in math, starting 
from the debate on international literature until the definition of the FA 
construct used in the FAMT&L Group. In particular, the authors analyse 
the aspects of the effectiveness of the FA in student learning. 

The third chapter presents in detail the FAMT&L Project and all the 
steps that have been taken for its implementation: from the analysis of the 
training needs to the implementation and monitoring of the pilot training 
course for teachers (based on the video analysis methodology). 

The fourth chapter presents the survey that has been carried out on 
teachers’ and students’ beliefs regarding the FA. It describes the data 
collection system, the tools used, and the main results that have led to the 
identification of teacher training needs. 

The fifth chapter deals with the methodologies of video analysis for the 
professionalism of teachers; the theoretical framework is presented and the 
basic techniques are described. 

Chapter 6 describes the specific observation and video analysis method 
used within the FAMT&L Project. Web repository and grids for classroom 
assessment analysis are presented in their core aspects. 

The seventh chapter deepens the training courses curriculum and it 
describes the design of the monitoring; for example, the results of the 
Italian experimentation. The synergies between researchers and teachers 
are highlighted. 

The eighth chapter directly collects the voice of the teachers who 
participated in the Italian pilot course. Teachers describe their experiences 
and skills acquired, pointing out the strengths and weaknesses of 
experimentation. 

Ninth and tenth chapters account for two fundamental aspects of the 
FAMT&L Project: the quality assurance procedures used and the exploitation 
strategies. These are two strictly technical chapters that help you get in the 
way of how the FAMT&L Group has consistently sought to evaluate, adjust 
the course and open up opportunities for further development. 
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Thanks to the Publishing House Franco Angeli of Milan, and in 
particular to Katiuska Bortolozzo, for having believed in this project. 
Giving voice to the processes and results of this project, we hope the book 
will be useful to the international debate and research on the development 
of teachers’ professionalism. 
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1. FAMT&L: 
a Teacher Professional Development Research 


by Giorgio Bolondi*, Ira Vannini** 


1. Origins of the FAMT&L Project 


The FAMT&L Project was born out of a firm belief and a key 
question. Firstly, the firm belief that formative assessment is a potentially 
revolutionary element for the educational and teaching function of the 
school systems, because it based on the idea of a democratic school (equity 
and quality together). Specifically, formative assessment is based on the 
bloomian belief (Bloom, 1972) that all students can learn basic skills in 
each topic and can do it excellently. 

Later, the key question arises of how it is possible to accompany and 
support teachers in acquiring actual assessment skills; or rather: how to 
orient their practices towards formative assessment? This question involved 
a group of educational and mathematics researchers from five European 
countries (Cyprus, France, Italy, Holland and Switzerland), who have defined 
a research project that has involved, as protagonists, schools and teachers. 

Within educational research, the research question on teachers’ 
professionalism is accompanied by a broad and complex debate, which also 
calls on disciplinary experts and all those concerned with teachers training 
processes (pre-service and in-service). The key question is what are the 
most effective contexts and strategies to support the change of the teachers; 
a change towards: 

e pedagogically founded and scientifically validated knowledge; 
e skills and beliefs consistent with a school ideal of equity and the quality 
of learning outcomes of students. 


* Free University of Bozen. 
** Alma Mater Studiorum Universita di Bologna. 
Chapter 1: Giorgio Bolondi; Chapters 2 and others: Ira Vannini. 
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In this sense, the international debate refers to the concept of Teacher 
Change (Floden, 2002; Guskey, 2002), a line of research devoted to 
exploring the processes of change (or resistance to change) that teachers 
put in practice during the teacher education or teacher training and staff 
development paths. 

The FAMT&L research group shared the great importance of this 
theoretical framework and focused on the teachers change in the assessment 
skills; and specifically: 

e What does it mean to support teachers in training processes oriented 
towards authentic practices of formative assessment in classroom? 

e What are the most effective training strategies? What strategies allow a 
real change in the teachers’ beliefs and in the teachers’ practices? 

e Which training tools allow to foster the teachers’ reflective thinking 
(considered as a fundamental characteristic of teacher professionalism)? 
These questions have become the central idea around which the group 

of researchers and teachers worked for three years; the partnership has 

been particularly profitable, especially because it has exploited the specific 
skills of: 

e educational researchers, experts in the field of assessment and 
evaluation processes and empirical research on teachers; 

e researchers in mathematics teaching, a field of advanced studies on the 
strategies of teacher training (D’Amore, 2009; English, Bartolini, et al. 
2008), and which put the focus of attention on the urgency of teacher 
professionalism in mathematics teaching; 

e school teachers associated with the Project, who had already considered 
and worked on the formative assessment in classroom and were 
available to give oneself a challenge in their teaching and assessment 
skills. 

The research path has been structured in a complex way and (as will 
be described in the subsequent chapters) has followed several phases, each 
closely linked to the others: 

e the analysis of the training needs of the teachers through sample 
surveys (in each of the five countries) on teacher’ beliefs and through 
the video observation in classroom; 

e the development of a specific observation tool for the videoanalysis of 
classroom assessment practices; 

e the design, implementation and monitoring of pilot training courses 
for the use of formative assessment, supported by the videoanalysis 
technique. 

During each design phase, the style of work was characterized by 
the comparison and collaboration between researchers and teachers; the 
group worked on the logic of sharing theoretical-value choices, objectives 
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and methodologies, complementarity of skills, professional growth of all 
actors. 

In this perspective, the path can be considered a true “Teacher 
Professional Development Research” (in Italian language: Ricerca- 
Formazione, R-F). 


2.An international path of “Teacher Professional 
Development Research” (R-F) 


The term “Ricerca-Formazione (R-F)” is used in the Italian educational 
debate, although it has not acquired a shared definition among the various 
scientific currents. At international level, there is no real literal translation. 
The conceptual translation in English language could be “Teacher 
Professional Development Research”. 

The term R-F refers to a not well defined concept within the scientific 
debate in education; it can be traced back to different research traditions. 
First of all, the R-F refers its origin to the participating research (Mantovani, 
1998) and to the more structured model of the Action-Research (Lewin, 
1946), where the key elements are the sharing of researchers and actors 
and their commitment to change and social action. In the second instance, 
the concept of R-F was used in the current called Formative Educational 
Evaluation (Becchi, Bondioli, 1994; Bondioli, Ferrari, 2004) that considers 
the evaluation (of the contexts, curricula, programs) as an opportunity for 
self-assessment of actors actively involved in the processes and for their 
professional growth. Furthermore, in several cases, R-F is also considered 
as a process of individual professional growth (eg through autobiographical 
analysis tools). 

Within the CRESPI Center (Center of educational research on teacher 
professionalism http://crespi.edu.unibo.it/) of the Alma Mater Studiorum 
Universita di Bologna, a theoretical idea of R-F has been discussed and 
defined starting from empirical research experiences carried out by 
CRESPI researchers in the field of teacher professionalism. The path of the 
FAMT&L Project can be related to this idea of R-F. 

CRESPI considers the R-F as a real empirical research that is 
carried out in the field of teacher professionalism in order to promote its 
development. It is a research that takes place within school institutions, 
strongly anchored to the context, where researchers and teachers share 
research itineraries but above all the goals of institutional change. Rather 
than a methodological choice (each R-F is careful to guarantee the 
validity and reliability of the collected data by adopting specific validation 
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procedures depending on the designs — qualitative, quantitative or mixed — 

that are applied) is a stance of researchers towards the school context and 

its teachers. 

Those who engage in an R-F aim at effectively affecting the 
professional development of the teachers involved and, at the same time, of 
the school context. Research is carried out following deliberate decisions 
in the group: preferably an interdisciplinary community of researchers in 
collaboration with teachers, which discusses and clarifies the framework of 
educational and political-social values of reference. 

«In the CRESPI Center (CRESPI Academic Group, 2016; http://crespi. 
edu.unibo.it/centro), the following methodological characteristics can be 
considered both as binding principles for researchers in the field of R-F 
and open questions in search of answers. 

1. Render explicit the aims of the research in terms of the professional 
learning and development of the teachers involved together with the 
procedures for documentation and evaluation of the outcomes and 
impacts. 

2. Define the composition of the TPDR group in terms of researcher/s 
and teachers, clarify their roles and negotiate and specify objects and 
objectives, values and methodological choices. 

3. Delineate and maintain a clear focus on the specificity of the 
institutional and non-institutional contexts in which the research 
is carried out through constant analysis of the constraints and the 
resources present and which are crucial in all phases of the research. 

4. Systematically discuss ways of experimenting and gathering data during 
the research, so as to enhance the development of a scientific approach 
to educational praxis on the part of teachers, and of evaluating and 
documenting products and processes within the specific contexts 
involved and the professional development promoted. 

5. Maintain emphasis on the outcomes achieved by and for the school 
in terms of educational innovation, teaching praxis and professional 
development». 

Below is discussed each R-F characteristic in function of how it was 
addressed within the FAMT&L Project. 


2.1. To clarify the aims of the research in terms of professional 
development, document and monitor them 


International framework of the FAMT&L Project has strongly 
contributed to the creation of a work style of sharing objectives, monitoring 
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and documentation. During the three-year of work, the R-F group had the 
purpose of promoting the development of teacher professionalism in the 
field of assessment practices. The main aim of the Project — after a phase 
of careful analysis of training needs and planning of tools and procedures 
— was to implement specific training pathways to change the beliefs and 
assessment practices of secondary school teachers by making them more and 
more adherent to the concept of Formative Assessment (see the Formative 
Assessment definition in FAMT&L group in the Chapter 2). 

This goal has been the guiding principle of all phases of the Project, 
helping the R-F group to maintain common care on the development of 
teacher professionalism. 

In addition, at each phase of the Project, the data collection was 
aimed at monitoring the processes in progress (analysis of teacher 
training needs through sample surveys, building and validation of tools 
for the observation and videoanalysis of practices, experimentation of 
training interventions by a pilot course in each country) and document 
— as transparently as possible — the results achieved. The purpose was to 
provide useful documentation not only to the R-F group, but as a point 
of reference to continue the research in other national and international 
contexts and to implement the pilot courses developed (training for 
Formative Assessment practices through methodologies of video analysis 
in classroom) on larger groups of teachers. 


2.2. Define the composition of the R-F group, clarify the roles 
and negotiate objectives, values and methodological 
choices 


As has been said, the R-F group was formed by the researchers together 
with a group of teachers from the five European countries participating in 
FAMT&L Project. The initial choices were elaborated and discussed in the 
academic group; from time to time, the goals and objects of the project 
were brought within the group of teachers to be deepened, rediscovered, 
further enriched. The ways of sharing within the group were different 
depending on the different work phases: 

e during the initial phases of training needs analysis, teachers were 
involved in discussion on the results of surveys (regarding both 
teacher’s beliefs data and practices observed in classroom). Within the 
several associated schools of the five European countries, and during 
meeting group, the researchers presented the beliefs and practices 
data to the teachers, and their first interpretations. Along with the 
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teachers’ groups, critical data interpretation was further analysed in 

specific school contexts; the data from the observations were analysed 

together with the same teachers video-recorded and fostering their self- 
evaluation processes; 

e in an explorative phase, the R-F group designed the videoanalysis 
tools and the curriculum of the pilot courses; in this time, the 
teachers have been actively involved. Initially, the grid for observing 
classroom practices was elaborated by the researchers from the 
Formation Assessment construct; subsequently this grid has been 
subjected to various try out in each country, through video analysis in 
the classrooms of the associated schools. Each country has suggested 
changes and improvements, up to the revision presented here which still 
constitutes a hypothesis tool open to further improvements (possibly 
through further discussions with teachers in new school contexts). 
The design of the pilot courses and the procedures to accompany the 
teacher’s reflexivity has been different: in this case, the researchers 
group proceeded independently from reference literature and specific 
academic experiences in the field of teacher training; 

e in the pilot phase, teachers were involved in a continuous process 
of monitoring and evaluating courses, both through systematic tools 
and through more qualitative procedures such as focus groups and 
interviews. These procedures have allowed teachers to take part as a 
protagonist in courses, to reflect critically on the knowledge and skills 
they were gaining and on the training methodologies adopted. During 
this time, also the video analysis grid was further subjected to the 
screening of teachers. 

As it can be seen, the R-F group has not provided roles of parity 
between researchers and teachers, but of complementarity and constant 
exchange between academics and school actors: processes and results 
have been constantly analysed, questioned and re-designed between them. 
In roles negotiation, teachers were considered the earliest recipients of the 
project’s results, the researchers were considered the first responsible for 
such results. 

This negotiation was initially shared with the leaders of the associated 
schools of the countries involved. 


2.3. Analysing the institutional constraints and resources of 
the R-F contexts 


As mentioned in the previous point, the associated schools have been 
largely involved in all phases of the Project. This involvement has led, 
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from the very beginning of the Project, a commitment of researchers to 
enter within the institutional contexts of the associated schools of the 
five countries. In fact, each school context has brought with it important 
resources, but also constraints that each country has had to take into 
account in the various phases of the R-F. 

At the beginning of the project, it was necessary to identify the teachers 
of each school and this required an early access by researchers into schools 
and a good negotiation with school leaders. 

It is important to note that each school had already experience in 
the field of assessment teachers’ skills and was therefore sensitive to the 
subject; however, in some schools the teachers were able to choose for 
them to participate in the project; in others the leader was instructed 
to indicate the teachers. This involved, in each of the five countries, an 
initial work for the constitution of the group and a progressive integration 
of the teachers within the R-F group. The care of relationships has been 
important in all five European countries, in the awareness that the socio- 
cultural context is a key element in order to carry out the effective R-F 
processes. 

The most important institutional constraints have been met on the front 
of privacy, and at different levels. 

First, during the surveys on teacher’s beliefs and student’s beliefs. In 
these two occasions, the schools in the various countries have proposed 
their specific regulations to allow data collection, both among teachers, but 
especially among students who are not yet of age. 

Subsequently, the most important constraint was placed during class 
video-recording. On the one hand, classroom video tutorials involved 
collecting images related to students as well. In compliance with the 
national privacy laws, they have been signed with family liberators; 
however, in Holland — despite all the formal reassurances — it was not 
possible to enter in the classroom with a camcorder. 

On the other hand, the video has created resistances in the same 
teachers involved in the R-F group, dictated by fears and lack of self- 
efficacy. It was therefore necessary to identify specific solutions within 
each institutional context: almost always the researchers increased the 
reflection meetings with the teachers in order to deepen the meaning of 
what was being done and to design together the assessment events to 
be filmed, while always guaranteeing the authenticity of the teachers’ 
actions. 

The analysis of constraints and institutional resources has been the 
subject of constant debate within the R-F group in order to identify the 
most sustainable solutions and to fully exploit all the theoretical and 
practical skills available within school contexts. 
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2.4. Evaluating and documenting products and processes: a 
systematically dialogue on the gathering data in the R-F 


group 


Each phase of the project was developed through a specific research 
design: a survey of teachers’ beliefs and a systematic observation 
survey during the early stages of the project; a qualitative-exploratory 
investigation for the development of video analysis tools; a pre- or quasi- 
experimental design to monitor the results achieved with the pilot courses. 
In each of these research moments, the R-F group has systematically 
documented the processes implemented: conversation and discussion among 
researchers, dialogue with teachers, decision-making, implementation of 
different procedures. This documentation has been facilitated by the fact 
that, within a European project, the research team is constantly urged to 
document using a variety of tools: minutes of meetings, e-mails between 
participants, drafting of reports in progress, to the specific documentation 
within the project’s dedicated website (www.famt-l.eu/it/). This involves a 
constant dialogue within the group and a constant exchange of information. 

The documentation of the results was achieved thanks both to research 
reports and to various publications in scientific journal that the researchers 
were writing during the project. 

What is to be noted, as a weakness, is the lack of in-depth after- 
sales analysis on the produced documentation. Whether the choices on 
what and how to document were the result of shared analysis within the 
R-F group; they have missed moments to go back to the documents and 
reflect critically in a re-design perspective. This in-depth analysis was done 
only at the end of the Project, especially for the preparation of the final 
conference and, above all, in order to imagine possible exploitation phases 
of the research and the ways to maintain (after the end of the project) the 
tools (in particular, the web repository). 


2.5. Maintain emphasis on the outcomes achieved by and 
for the school in terms of educational innovation and 
professional development 


The FAMT&L project, since its starting as a Comenius Project 
proposal, has consistently been geared towards identifying hypotheses and 
solutions for teacher training and their growth in terms of professionalism. 
Specifically, as has been said, the Project has always been geared towards 
changing teachers’ beliefs and formative assessment practices. As we 
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will see in the subsequent chapters, the R-F group’s work was based 
on a key idea: student learning assessment is a major issue in school 
systems across Europe; teacher skills are often lacking and the assessment 
practices are often used for selective purposes rather than for learning. 
This is particularly true in secondary schools and, specifically, in those 
school levels for students aged 11 to 16 who are at the end of compulsory 
education. It is in these levels that the highest rates of school dispersion 
are recorded and a strong demotivation to students learning. International 
educational research, especially since the ’90s (see references in the 
Chapter 2, shows that good formative assessment practices in classroom 
are largely effective to improve the students’ learning, to stimulate their 
self-assessment and their motivation for learning. Formative assessment, 
at the same time, enables the implementation of different teaching 
methodologies and leads the teacher to self-evaluate and to reflect critically 
on his educational choices. Result is a growth of teacher professional skills: 
curriculum designing, using of good teaching methodologies, measuring 
and assessment, managing formative feedback. 

From these considerations, the FAMT&L group has defined the main 
objective of the Project: to identify pathways and training arrangements 
to improve teachers’ assessment skills. The relapse of the Project on the 
professionalism of teachers had to be tangible, both in terms of effective 
change in the beliefs and practices of teachers involved in the R-F group, 
as well as in modelling pathways and training methodologies to be 
reopened within wider contexts. 

The results of the pilot courses, as can be seen in the following 
chapters, are encouraging: significant changes have been observed during 
pilot courses. Additionally, teachers in the 5 R-F groups are currently 
continuing to collaborate with university researchers and have been 
involved as trainers for new groups of teachers. 

Curricula designed for pilot courses — their training structure, video 
analysis methodologies, video materials, and tools for teacher reflexivity 
— are also a major result in terms of innovation in the programs for in- 
service teacher training. Now, we have to continue the exploration of our 
research hypothesis in wider contexts. 
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2. Why formative assessment in Mathematics? 


by Andrea Ciani*, Ira Vannini*, Federica Ferretti** 


1. Introduction 


Assessment in classroom has always been a key tool in order to 
promote, or to hinder, democratic values at school. An education system 
that does promote quality and equity for the learning achievements of 
its students, uses assessment as a key element to qualify the action 
of teaching in a democratic way, both at the beginning and during 
the process of teaching-learning; moreover it will consider the 
differences among the students and their possible learning difficulties 
as opportunities to make the teaching actions flexible in order to reach 
goals of quality for all (Vertecchi, 1976; Grandi, 1977; Weeden, Winter, 
Broadfoot, 2002). 

As we can read in Crahay (2013), it has to be a kind of assessment 
which adheres to a principle of equality of achievements (Bloom, 1968; 
Black, William, 1998; Guskey, 2005), hence to an idea of “fairness” in 
teaching, by offering more to whom possesses less. 

This need of fairness in achieving the competences for citizenship 
(OECD, 2015; Eurydice, 2012) is more evident in every education 
system when considering basic competences and at high and junior high 
school level, before the completion of the compulsory cycle of studies. 
In particular, relevant problems appear in the field of math teaching, 
with important gaps in the conduct of the specific teaching-learning 
processes. 


* Alma Mater Studiorum Universita di Bologna. 

** Free University of Bozen. 

Chapters 1 and 2: Ira Vannini, Chapter 3: Andrea Ciani, Chapters 4 and 5: Federica 
Ferretti. 
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2. Formative Assessment: assessment for learning for all 
the students 


Since its origins (Scriven, 1967; Vertecchi, 1976), the main function 
attributed to formative assessment (FA) is to be a regulator tool for 
teaching and learning. 

Referring to the current international scientific debate on this issue, 
we can say that formative assessment is characterized specifically as an 
assessment for learning (Weeden et al., 2002; Allal, Laveault, 2009). This 
means that it has to be an assessment which is functional to backing up and 
promoting learning; it is embedded in the teaching-learning process in a 
dynamic way, modifying the teaching actions by following the needs of the 
students. The aim will never be to just attribute marks, or to make a résumé 
on the abilities of a student; formative assessment helps a teacher to gather 
information to improve and to get her/his teaching action more effective. 

Thus, when a teacher uses formative assessment, s/he is implementing 
two fundamental actions (Vertecchi, 1976): 

e a diagnostic analysis of the achievements (knowledge, abilities) that the 
student is acquiring and of which meta-cognitive strategies the student 
is following; 

e a reconstruction of the teaching path by following the student’s needs 
and differentiating times and methods of the didactic process. 

Thanks to this diagnostic function, formative assessment analyzes the 
learning situations and can give information in order to take coherent and 
effective decisions. It focuses on the “errors” of the student and of the 
teacher, by considering them as resources for designing and re-designing 
interventions in view of the teaching goals. 

This kind of assessment requires a high professionalization of the 
teacher, which has to implement a continuous assessment attention during 
the teaching process, as a coach in the training of an athlete or a team 
(Bennet, 2010; 2015) who proposes activities and tasks to the trainees (as 
a trial for their abilities), detects and immediately corrects their errors (by 
discussing with the trainees about them), understands the specific needs 
and gives formative feedback. 

As every assessment procedure (Gattullo, 1967), also FA is 
characterized by three steps (Gitomer, Zisk, 2015, p. 3): 

e an initial step of cognitive representation of which data we want to 
collect (“what we are trying to measure”); 

e a step of specific gathering of data, by empiric observation (“how we 
collect evidence”); 

e the interpretation of the data (“how we make sense of the evidence”). 
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Collecting evidence is an unavoidable phase (Ruiz-Primo, Furtak, 
2004) inasmuch as it characterizes FA as a specifically evaluative action, 
both when it is formally or informally done (informally as in the course 
of a teacher’s day-to-day activities) (Bell, Cowie, 2001; Duschl, 2003; 
Shavelson et al., 2003). Without a willing gathering of evidence we would 
not be doing FA, but just a teaching activity. 

The next step, data interpretation, is equally important. Doing 
summative assessment this step would end in the attribution of marks 
or of a judgment, doing FA any judgment is suspended. It is formative 
feedback that must take place in this moment, instead: the teacher’s answer 
to the needs/requirements of the student. Researches show that feedback 
— together with FA — is the crucial element, which contributes, in a 
statistically significant way, to improve the results in the students’ learning 
(Hattie, Timperley, 2007; Hattie, 2009; 2012; Huelser, Metcalfe, 2012). 

For this reason the teacher’s practices in class are particularly 
important, both in the moment when data about the students’ achievements 
are gathered and analyzed, and when interpreting the data, elaborating 
hypotheses about the kind of mistakes the students do and implementing 
feedback actions to help them in the critical steps in their apprehension. 
All this is really fundamental in the teaching of mathematics. 

The feedback activity is a complex set of actions by the teacher, not 
easily described by a set of rules or given operations; for this reason 
researches in this field are particularly relevant and compelling: it is 
necessary, in fact, to clear up, in detail, which are the most effective 
conducts that the teacher has to implement in classes activities when facing 
a “stumbling” student. 

Via the feedback, the teacher should manage to make the student’s 
errors explicit, and make them valuable as an asset in the learning process; 
in the meanwhile the teacher has to sustain the students’ motivation 
to learn and to mobilize all their meta-cognitive strategies in order to 
overcome the obstacles. Here the didactic mediation is substantial; the 
teacher must use several and differentiated didactic tools, give additional 
explanations, sustain the students’ aloud reasoning (Weeden, Winter, 
Broadfoot, 2002; Bennet, 2010; Doabler et al., 2014). 

Several researches about teachers’ behavior highlight that they agree 
about the fundamental role of FA for the quality of teaching, nevertheless 
— in their practice — they follow more often summative assessment praxes. 
Also, in spite of the fact that they use FA in their ongoing activities, 
they may use superficial tests, propose mechanical answers, or give a 
feedback that is too generic (Looney, 2011, p. 10). In fact, they seem not 
to be prepared to interpret the evidence they have about their students’ 
learning and often they attribute to external reasons the impossibility to 
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implement FA (too many students in their classes, too large curricula to 
teach, organizational difficulties in their schools) (OECD, 2005). 


3. International debate: different views on the formative 
assessment 


The formative assessment, as it has been outlined since its inception, 
has been conceived, imagined, designed to increase students’ learning. 
Black & Wiliam studies and researches’ have allowed to spread and share 
in the world an idea of FA and to support the efficacy of its practice. 
Although Black & Wiliam works’ was referring to local contexts, data, 
considerations, it became a fundamental basis for the comparison and 
development of the international debate on the FA. 

The international debate on the FA is based on additional interpretations 
or new approaches, without contradicting its initial conception even if the 
educational perspectives of application could be radically different (e.g. FA 
in behaviourist or in social-constructivist view). 

In general, as Torrance (2012, p. 324) has documented, over the 
years the FA has defined more its meaning and the spectrum of related 
practices: 


from mastery learning and criterion-referenced graded assessment programmes in 
the 1970s and 1980s (Bloom, 1974; Popham, 1978; Pennycuick &Murphy, 1988); 
through systems approaches to feedback (Ramprasad, 1983; Sadler, 1989); to the 
variety of approaches to recording, reporting and profiling achievement which 
emerged in the 1980s and 1990s in the UK (Broadfoot, 1986; Torrance, 1991; Pole, 
1993). Currents of formative assessment, also known as ‘assessment for learning’, 
are often linked to Black and Wiliam’s (1998a) review of the literature, subsequent 
developments by the UK Assessment Reform Group (2002) and associated work by 
Black and colleagues. 


If initially, in the U.S., early interest tended to revolve around mastery 
learning e and formative assessments’ practices to develop programmes 
of ‘measurement-driven instruction’ (Popham, 1987; Airasian, 1988), over 
time a wider vision has been established, associating it with the assessment 
for learning. 

Stiggins (2005), one the first promoters in North America of this vision, 
explains how assessment for learning is an approach to the formative 
assessment and defines its differences with a traditional view. In its 
traditional form, formative assessment has been thought of as providing 
teachers with more frequent evidence of students’ mastery of standards to 
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help teachers make useful instructional decisions. In this way, formative 
assessment is intended to enhance student learning. 

The purpose is using a lot of different evaluations methods to provide 
students teachers and parents with a continuous flow of tests of the 
student progress in mastering competences that support or guide to set the 
standards: during the learning, students are inside the assessment process, 
watching them grow, and feeling in control of their success. The most 
important difference between the traditional approach and assessment for 
learning is that the former intend to inform the teachers about student 
achievement, while the latter also wants to inform students about their own 
learning. 


Assessment FOR learning rests on the understanding that students are data-based 
instructional decision makers too, a perspective all but ignored in our assessment 
legacy and in previous approaches to school improvement. Another difference 
is that traditional formative thinking tends to want more frequent assessment of 
student mastery of the standards themselves, while assessment FOR learning 
focuses on day-to-day progress in learning as students climb the curricular 
scaffolding leading up to state standards (Stiggins, 2005, p. 328). 


It is precisely on the guiding principles of assessment for learning that 
different ways of viewing the FA are born. In fact, in a similar way to what 
happens in the US, the initial conception of FA proposed by Bloom has 
been enlarged with other elements, in particularly 


In the enlarged perspective of formative assessment developed in French- 
language publications, the idea of remediation of learning difficulties (feedback + 
correction) is replaced by the broader concept of regulation of learning (feedback 
+ adaptation). In an enlarged conception, external regulation (by the teacher, by 
the test, by remedial material) is redefined as scaffolding that assists students’ 
development of self-regulation. [...]This means fostering the active involvement of 
students in formative assessment through procedures of self-assessment, reciprocal 
peer-assessment, and joint teacher-student assessment (Allal, 2005, p. 245). 


In the French FA enlarged perspectives, the learning regulation 
(interactive, retroactive and proactive) is fundamental. Allal writes that 
concept of regulation in the French-language literature about FA is linked to: 
e the degree of active student involvement in these actions; 

e the ways students make use of tools and resources present in the 
instructional environment to adapt or enrich their learning activity; 

e the meaning the students and teachers give to the various aspects of 
assessment; 

e the ways in which teachers and students negotiate assessment. 
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Thus, vision that emerges in the French pedagogical debate supports a 
socio-constructivist perspective of FA, close or almost superimposable to 
the approach of assessment for learning. 

Although in Germany the FA in a Bloom/Black&Wiliam sense is not 
consolidated yet and there are few FA studies in comparison with UK, US 
and France, the alternative education has emphasized that teachers should 
be aware of how they provide feedback to students, as feedback indicating 
personal growth to students could foster their learning and motivational 
development (Kôüller, 2005). 

In Italy, as in several other countries, the effort to promote a FA culture 
in schools following the Bloom perspective is quite substantial, above 
all in universities and in the world of educational research, but is also 
increasingly combined with the attention to the teaching, motivational 
and communicative processes. For this reason, in Italy people talk more 
and more of FA following the assessment for learning approach and the 
FAMT&L research has adopted this term to indicate a complete and 
upgraded view of FA. 


4. Formative Assessment in Mathematics 


The important role of the assessment is underlined in most National 
Curricula and it is entrusted to teachers, individual schools and ministerial 
institutions. The assessment is under the responsibility of the teacher in all 
the phases, both of the planning and of the operational process of teaching; 
for instance, in Italian National Curricula, the formative function of 
evaluation is clearly emphasized, as are its crucial role inside the learning 
process and its function as continuous stimulus for improvement. 

Formative assessment for learning should indeed be essential part in all 
phases of the process of teaching and learning. From this perspective, its 
main function is regulative and its main objective is to help teachers and 
students to continuously adapt the teaching/learning process, especially 
if it is oriented to the individualisation of teaching procedures (Tornar, 
2001). Specifically, formative and diagnostic function of evaluation is 
carried out, referring to a formative path, ex ante and in itinere with the 
aim of collecting detailed information both at initial stages and during 
the process of learning, whenever students are faced with difficulties, 
so to design educational targeted interventions and to have a constant 
and reciprocal feedback between teacher and pupil. Hence the purpose 
of formative assessment is essentially educational and its main aim is to 
provide feedback and information to the teachers, so that they can then 
do something to promote their students’ learning. Formative assessment 
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is also characterized by the fact that it does not provide the explanation 
of a judgment or a vote, but is rather characterized by a significate 
enhancement of errors, considered as a resource to improve the quality of 
the educational path (Gagatsis & Christou, 1997; Gagatsis & Kyriakides, 
2000, Zan, 2007). Some studies have allowed us to collect empirical 
evidence that could prove, for example, that the systematic use of the in 
itinere evaluation of the students’ progress generates significant increases 
in their final performance. Moreover, the quality of the feedback, the 
active participation of the students in the evaluation process, some aspects 
of verbal interaction in the classroom and the effects of evaluation on 
self-confidence and motivation, have shown themselves to be crucial for 
the effectiveness of formative assessment to promote learning (Scallon, 
1985; Black & William, 1998). From these studies a discussion at the 
international level has developed, getting to the conclusion that formative 
assessment is an “assessment for learning” (Weeden ef al., 2002). 


5. Why formative assessment in Mathematics? 


Teaching and learning processes concerning mathematics and 
sciences are a fundamental component of school activities, and they are 
preliminary to many of the skills that are significant in life and necessary 
for the citizenship formation. Mathematics paves the way for different 
ways of thinking and for applications that affect daily life, allowing critical 
interpretation and assessment of the huge amount of information produced 
in the modern learning society. Moreover, it also determines the practice 
of citizenship through logical reasoned and motivated decision-making 
processes about social issues. 

Without adequate competence-building in mathematics (thinking, 
logical reasoning, etc.) it becomes virtually impossible to fully access the 
contemporary world made by information, communication and technology. 
This affects the opportunities for all to be involved in social and economic 
life. This idea is also stressed in the concept of “mathematical literacy”, 
as defined in the OCSE Programme for International Student Assessment 
— PISA (OECD, 2013) as the capability to identify, understand and engage 
in mathematics, and to make well-founded judgements about the role 
that mathematics plays in an individual’s current and future private 
life, occupational life, social life with peers and relatives, and life as a 
constructive, concerned and reflective citizen. 

Despite researchers’ and teachers’ engagement, the crisis in mathematical 
education & learning is becoming very diffused and profound. Mathematics 
tends to be seen by young and adults as an uninteresting discipline. 
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According to recent international research (OECD, 2014; Eurydice, 
2012), the main difficulties in mathematics learning are represented by: 

e a severe lack of mathematical skills among students aged 15th in many 
European countries (OECD, 2005) based on problems in mathematical 
didactics starting from the 11-14/15 years age group; 

e teaching methodologies and pedagogic lack in giving a broad “sense 
of number” and the ability to work with it (with figures, measures, 
statistics and probability); 

e incorrect use of formative assessment and need to introduce strategies 
of teaching and learning individualization; 

e in general, rising de-motivation for school learning, starting especially 
in the 11-14/15 years age; 

e more specifically, de-motivation for learning mathematics. 

The 11-14/15 years age group appears to be the one that requires 
innovative and adequate interventions to improve mathematical and — 
especially — numeracy skills, reclaiming both to mind for the lost ground in 
mathematical curiousness amongst youngsters as well to bring all students 
to obtain satisfactory results with a better and broader commitment to 
mathematical reasoning in any social-economic application. 

It appears to be important to design innovative ways to invest in teacher 
training for mathematics, in particular through training paths that put 
teachers in collective planning situations and bring them to build plans, 
strategies and methodologies for teaching mathematics effectively. It goes 
without saying that this does not only means to investigate on the subject, 
but also on the pedagogical-didactical learning strategies, in particular 
in the field of assessment, stressing the value of formative assessment 
and individualized teaching strategies, planned to specifically respond to 
students” rhythm and learning styles. 

For this reason, it was very important to investigate on mathematics 
teachers’ beliefs and misconceptions about assessment in the classroom 
and to analyse learning activities in the classroom, investigating on 
teachers’ rationales behind learning difficulties in mathematics in order to 
plan adequate interventions for remedial programming. 

At the same time, we think that each teacher should acquire the 
necessary competences for reflecting on the content of mathematical 
education as well as on the pedagogical-didactical aspects. This implies to 
have an internal focus on mathematics itself and an external focus on the 
links of mathematics to other subjects. In both cases, it is important for 
teachers to learn, assess and diagnose students’ learning needs and to give 
answer to them with adequate teaching methodologies. 

During all project activities and implementations, the mathematics 
has been seen and validated as a multi-functional and multi-disciplinary 
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subject in school. One of the fundamental principles on which we are 
based is that the teachers need to be aware about their crucial position in 
school and about their need to reflect on difficulties and mistakes, to find 
the causes of them and to plan the interventions for remedial programming 
through efficacy strategies and tools of formative assessment. 

In this perspective it is therefore necessary to train mathematic teachers 
not only in mathematics and didactics of mathematics, but also in the 
application of these disciplines in their own context. From this point of 
view, the solution lies mostly in the hands of teachers and teacher- 
trainers, more specifically in designing and giving them the instruments of 
educational planning and assessment. 


6. FAMT&L view on the Formative Assessment 


In 2008, Sattler has defined assessment as a way to understand a 
student, in order to make informed decisions related to classroom practices 
that involve him. In most theoretical frameworks for evaluation the first 
step of the evaluation process it is the diagnostic assessment, but it is quite 
of common knowledge that the evaluation process, and in particular the 
role of diagnostic and formative assessment, is often based on mechanical 
procedures influenced by implicit philosophies of each teacher, in 
particular in mathematics (Speranza, 1997). 

Since the °90s there has been a growing awareness about the need for 
significant changes in assessment practices of learning in mathematics and 
likely this is related to the fact that more and more mathematics has been 
identified as a key competence within the frame of life skills (Shepard, 
1989; Webb & Coxford, 1993). The evaluation process is considered as an 
integral part of the learning process (Desforges, 1989) and the importance 
attached to the assessment in mathematics is reflected both by the impact of 
increasing disciplinary national and international standardized assessments 
(see, for example, OECD, 2015) and by the research work about the 
practices of classroom assessment (DES, 1987; NCTM, 1989). 

In detail, as for learning mathematics, usually teachers assess 
knowledge, skills and abilities of students. In the school practice, 
assessment in mathematics is often divided and organized on the base 
of the content of teaching, although it is an increasingly shared notion 
that to properly assess cross components of learning is needed. In recent 
times especially there is an increasing interest on what competence in 
mathematics means and how it can be evaluated. FA is connected with a 
concept of learning according to which all students are able to acquire, at a 
adequate level, the basic skills of a discipline. Learning passes through the 
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use of teaching methodologies which can respond effectively to different 
learning time for each student, to different learning styles and to zones of 
proximal development. 

Basing on these references and results, we adopt the following 
definition of formative assessment: 


FA in the classroom is an assessment FOR teaching and learning. 

It is part of the teaching-learning process and regulates it; it identifies, 
in an analytical way, the strengths and weaknesses of student’s learning 
in order to allow teachers to reflect on it and maybe modify their own 
practices; it allows a formative feedback to establish a dialogue between 
teacher and student and to design educational interventions aimed to 
the recovery; it promote and foster the learning by all students through 
differentiated teaching that ensures each student different rhythms and 
different teaching and learning strategies; it involves the students in the 
analysis of their own errors/weaknesses and their own ability to promote 
self- and peer-assessment and active participation in the teaching- 
learning process. 

It is intended to give information, feedback and feed forward — in and 
outside of the classroom — related to the development of mathematical life- 
skills. In particular: 

e it addresses the different components and aspects of mathematical 
learning of the students (conceptual, procedural, communicative, 
semiotic); 

e it is involved in analyzing problem posing and solving strategies, 
misconceptions, organization of mathematical experience, students’ 
beliefs, students’ image of mathematics and of specific segments of 
mathematics; students’ behavior and classroom interaction when involved 
in different mathematical tasks; 

e it is a critical issue to make clear what the outputs of teacher’s choices 
are (transposition of mathematical contents, interface between contents 
and methods). 


References 


Airasian, P. (1988). Measurement-driven instruction: a closer look. Educational 
Measurement: Issues and Practice, 7(4), 6-11. 

Allal, L., Lopez, L.M. (2005). Formative assessment of learning: A review of 
publications in French. In J. Looney (Ed.). Formative assessment: Improving 
learning in secondary classrooms (pp. 241-264). Paris: Organisation for 
Economic Cooperation and Development. 


34 


Copyright © 2018 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788891774637 


Assessment Reform Group (2002) Assessment for Learning: 10 principles. Available 
online at: www.assessment-reform-group.org (accessed 24 August 2011). 

Ausubel, D.P. (1968). Educational psychology: A cognitive view. New York, NY: 
Holt, Rinehart & Winston. 

Bell, B. & Cowie, B. (2001). The characteristics of formative assessment in 
science education. Science Education, 85 (5), 536-553. 

Bennet, R.E. (2010). Cognitively based assessment of, for, and as learning. 
A preliminary theory of action for summative and formative assessment. 
Measurement; Interdisciplinary Research and Perspectives, 8, 70-91. 

Bennet, R.E. (2015). The changing nature of educational assessment. Review of 
Research in Education, AERA, 39, 370-407. 

Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in 
Education: Principles, Policy & Practice, 5, 7-74. 

Bloom, B. (1974) An introduction to Mastery Learning Theory. In J. Block (Ed.), 
Schools, society and Mastery Learning. New York: Holt, Rinehart & Winston. 

Broadfoot, P. (1986). Whatever happened to inequality? Assessment policy and 
inequality: the UK experience. British Journal of the Sociology of Education, 
7, 2, 2015-224. 

Crahay, M. & Issaieva, E. (2013). Conceptions de l’évaluation et principes de 
justice chez des enseignants primaires en Fédération Wallonie-Bruxelles. 
paper presented at ADMEE 2013 — Evaluation et autoévaluation. Quels 
espaces de formation? — Fribourg, 9-11 Janvier 2013. 

Department of Education and Science (1987), National curriculum: Task group 
on assessment and testing — A report. London: HMSO. 

Desforges, C. (1989). Testing and Assessment. London: Cassells. 

Doabler, C.T., Nelson, N.J., Kosty, DB., Fien, H., Baker, S.K., Smolkowski, K. & 
Clarke, B. (2014). Examing teachers’ use of Evidence-Based practices during 
core Mathematics instruction. Assessment for Effective Intervention, 39 (2), 
99-111. 

Eurydice (2012). Developing key competences at school in Europe: Challenges 
and opportunities for Policy. Brussels: Education, Audiovisual and Culture 
Executive Agency. Retrieved September 15, 2015, from: www.indire.it. 

Duschl, R.A. (2003). Assessment of inquiry. In JM. Atkin, JE. Coffey (Eds.), 
Everyday assessment in the science classroom (41-59). Washington, DC: 
National Science Teachers Association Press. 

Hattie, J. (2009). Visible learning. A synthesis of over 800 meta-analysis relating 
to achievement. London-New York: Routledge. 

Hattie, J., Tymperley, H. (2007). The power of feedback. Review of Educational 
Research, 77 (1), 81-112. 

Huelser, B.J. & Metcalfe, J. (2012). Making related errors facilitates learning, but 
learners do not know it. Memory and Cognition, 40 (4), 514-527. 

Koller, O. (2005). Formative assessment in classrooms: A review of the empirical 
German literature. In J. Looney (Ed.), Formative assessment: Improving 
learning in secondary classrooms (pp. 265-279). Paris, France: Organisation 
for Economic Cooperation and Development. 


35 


Copyright © 2018 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788891774637 


Gagatsis, A., Kyriakides, L. (2000). Teachers’ attitudes towards their pupils’ 
mathematical errors. Educational Research and Evaluation, 6 (1), 24-58. 

Gagatsis, A. & Christou, C. (1997). Errors in mathematics: A multidimensional 
approach. Scientia Paedagogica Experimentalis, 34 (1), 89-116. 

Gattullo, M. (1967). Didattica e docimologia. Roma: Armando. 

Gitomer, D.H., Zisk, R.C. (2015). Knowing what teachers know. Review of Research 
in Education, AERA, 39, 1-53. 

Grandi, G. (1977). Misurazione e valutazione. Firenze: La Nuova Italia. 

Guskey, T.R. (2005). Formative classroom assessment and Benjamin S. Bloom: 
Theory, research and implications, paper presented at the Annual Meeting of 
the AERA (American Educational Research Association). Montreal, Canada, 
April 11-15, 2005. Retrieved September 15, 2015, from: http://files.eric.ed.gov/ 
fulltext/ED490412.pdf. 

National Council of Teachers of Mathematics (1989). Curriculum and evaluation 
standards for school mathematics. Reston, Virginia: NCTM. 

OECD (2013). PISA 2012 Assessment and Analytical Framework. OECD 
Publishing. 

OECD (2014). 2012 Results in Focus: What 15-year-olds know and what they can 
do with what they know. OECD Publishing. 

OECD (2015). OECD Skills Outlook 2015: youth, skills and employability. Retrieved 
September, 2015, from: www.oecd library.org. 

Pennycuick, D. & Murphy, R. (1988). The impact of graded tests. London: Routledge. 

Pole, C. (1993). Assessing and recording achievement. Buckingham: Open University 
Press. 

Popham, J. (1978). Criterion-referenced measurement. Prentice-Hall, NJ: Englewood 
Cliffs. 

Popham, J. (1987). The merits of measurement-driven instruction. Phi Delta Kappan, 
68, 679-682. 

Ramaprasad, A. (1983). On the definition of feedback. Behavioural Science, 28, 
4-13. 

Ruiz-Primo, M.A., Furtak, E.M. (2004). Informal formative assessment of 
students’ Understanding of scientific inquiry. CSE — Center for the Study of 
Evaluation — Report 639, Los Angeles, CA. 

Sadler, R. (1989). Formative assessment and the design of instructional systems. 
Instructional Science, 18, 119-144. 

Sattler, J. (2008). Assessment of children: Cognitive foundations. La Mesa, CA: 
Jerome Sattler. 

Scallon, G. (1985). La participation des éléves au diagnostic pédagogique: 
exploration avec des élèves de 4° secondaire en mathématiques. Mesure et 
évaluation en éducation, 8, 5-44. 

Scriven, M. (1967). The metodology of evaluation. In R.E. Tyler, R.M. Gagné, M. 
Scriven, Perspecive of corriculum evaluation. Chicago: AERA Monograph 
Series in Education. 

Shavelson, R.J., Black, P., Wiliam, D., Coffey, J. (2003). On aligning summative 
and formative functions in the design of large-scale assessment system. Paper 
presented at the National Research Council’s workshop on Assessment in 
Support of Instruction and Learning, January 2003. 


36 


Copyright © 2018 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788891774637 


Shepard, L.A. (1989). Why we need better assessments. Educational Leadership, 
46 (7), 4-9. 

Speranza, F. (1997). Scritti di Epistemologia della Matematica. Bologna: Pitagora. 

Stiggins, R.J. (2005). From formative assessment to assessment FOR learning: A 
path to success in standards-based schools. Phi Delta Kappan, 87(4), 324-328. 

Thomson, C.J. & Davis, S.B. (2014). Classroom observation data and instruction 
in primary mathematics education: improving design and rigour. Mathematics 
Education Research Journal, 26, 2, 301-323. 

Tornar, C. (2001). Il processo didattico tra organizzazione e controllo. Roma: 
Monolite Editrice. 

Torrance, H. (1991). Records of achievement and formative assessment: some 
complexities of practice. In R. Stake (Ed.). Advances in program evaluation: 
using assessment policy to reform education (pp. 231-245), Greenwich, CT: 
JAI Press. 

Torrance, H. (2012). Formative assessment at the crossroads: conformative, 
deformative and transformative assessment. Oxford Review of Education, 38 
(3), 323-342. 

Vannini, I. (2012). Come cambia la cultura degli insegnanti. Metodi per la 
ricerca empirica in educazione. Milano: FrancoAngeli. 

Vertecchi, B. (1976). Valutazione Formativa. Torino: Loescher. 

Webb, N.L., Coxford, A.F. (Eds.) (1993). Assessment in the mathematics classroom. 
Reston, VA: National Council of Teachers of Mathematics. 

Weeden, P., Winter, J. & Broadfoot, P. (2002). Assessment. What's in it for schools. 
London: Routledge. 

Zan, R. (2007). Difficoltà in matematica. Osservare, interpretare, intervenire. 
Berlino: Springer. 


37 


Copyright © 2018 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788891774637 


3. THE LLP-Comenius FAMT&L Project 


by Federica Ferretti*, Athanasios Gagatsis** 


1. Introduction 


Teaching and learning processes concerning mathematics and sciences 
are a fundamental component of school activities, preliminary to most 
of the competences that are significant in life and necessary for the 
citizenship education. Nevertheless, despite researchers’ and teachers’ 
engagement, the crisis in mathematics learning is becoming very diffused 
and serious (OECD, 2015). 

In order to contribute to the development of lifelong quality learning 
and to promote high performance, innovation and a European dimension in 
systems and practices in the field, the Comenius Project aims to: 

e provide a methodological model and some valid materials for a quality 
proposal of mathematics teacher training in Europe-starting from an 
in-depth analysis of teachers’ needs and on-going experiences in the 
different partner countries. In particular, the training proposal, within 
a lifelong learning perspective, is addressed to in-service mathematics 
teachers in order to significantly enhance their assessment competences. 

e to promote in-service training of mathematics teachers through the 
design and implementation of a web repository of learning objects, such 
as video reporting significant classroom activities, assessment tools, etc. 
The repository will be an innovative product of the project, to be used 
in the future as an online resource for training activities addressed to 
mathematics teachers; 

e to help young teachers (students and pre-service teachers) acquire 
the basic life-skills and competences necessary for their personal 
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development, for future employment and for active European 

citizenship; 

e to contribute to the improvement of in-service mathematics teachers’ 
training through the design of specific on line support and the planning 
of pilot training courses, to be capitalized for the years ahead. The 
products of the project (repository and models training) aim to provide 
resources for the teachers training and staff development in Europe, 
adaptable to different contexts and always updated; 

e to improve the quality of mathematics education in Europe and, 
therefore, to increase the level of basic skills in mathematics for 
secondary school students. Focusing on the relationship between teaching 
and formative assessment gives an opportunity to promote metacognitive 
and transversal skills (such as monitoring and self-control of learning 
from the students) in mathematics teaching processes. 

The project, through some specific resources and actions for supporting 
in-service teachers training, aimed to improve mathematics teaching 
strategies for pupils aged 11-14/15. In particular, the focus on formative 
assessment methodologies in the classroom is crucial to make teachers 
really able to improve students’ mathematics skills. 

In details, the main goal of FAMT&L project consisted in realizing a 
training model (in blended modality) for middle school math teachers (that 
can be applied to in-service and pre-service training). This training model 
(or methodology) should improve teachers’ competences: 

e on educational planning and assessment (both formative and summative 
assessment; assessment for learning); 

e on mathematics education. 

As presented in chapter 6, teacher training goals are achieved. 


2. The main topics of the Project 


Teaching and learning processes concerning mathematics and sciences 
are a fundamental component of school activities, preliminary to many 
of the skills that are significant in life and necessary for the citizenship 
formation. Mathematics paves the way for different ways of thinking 
and applications that affect daily life allowing critical interpretation and 
assessment of the huge amount of information produced in the modern 
learning society. Moreover, it also determines the exercise of citizenship 
through logical reasoned and motivated decision-making on social issues. 

Without adequate competence-building in mathematics (thinking, 
logical reasoning, etc.) it becomes virtually impossible to fully access this 
modern world made up by information, communication and technology. 


39 


Copyright © 2018 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788891774637 


This affects the opportunities for getting involved in social and economic 

life. This notion is also stressed in the concept of “mathematical literacy”, 

as defined in the OCSE Programme for International Student Assessment 

— PISA (OECD, 2012) as the capability to identify, understand and engage 

in mathematics, and to make well-founded judgements about the role 

that mathematics plays in an individuals current and future private 

life, occupational life, social life with peers and relatives, and life as a 

constructive, concerned and reflective citizen. 

Despite researchers’ and teachers’ engagement, the crisis in mathematical 
education and learning is becoming very diffused and profound. Mathematics 
tends to be seen by young and adults as an uninteresting discipline. 

According to recent international research (OECD, 2012, Eurydice, 
2012), the main difficulties in mathematics learning are represented by: 

e a severe lack of mathematical skills among students aged 15" in many 
European countries (OECD, 2005) in relation to problems of the 
didactics of mathematics starting from the 11-14/15 years age group; 

e teaching methodologies and pedagogic lack for almost all education 
sectors, a broad “sense of number” and the work with it, figures, 
measures, statistics and probability; 

e incorrect use of formative assessment and need to introduce strategies 
of teaching and learning individualization; 

e in general, rising de-motivation for school learning, starting especially 
in the 11-14/15 years age; 

e more specifically, de-motivation for learning mathematics. 

The 11-14/15 years age group appears to requests innovative and 
adequate interventions to improve their mathematical and — especially 
— numeracy skills, reclaiming both the lost ground in mathematical 
curiousness amongst youngsters, as well as bringing all students to obtain 
satisfactory results with a better and broader commitment to mathematical 
reasoning in any social-economic application. 

Thus, designing innovative ways to invest in teachers’ training for 
mathematics appears to be important, in particular through training paths 
that put teachers in collective planning situations and bring them building 
plans, strategies and methodologies for teaching mathematics effectively. 
Of course, this does not only mean to investigate on the subject, but 
also on the pedagogical-didactical learning strategies, in particular in 
the field of assessment, stressing on the value of formative assessment 
and individualized teaching strategies, planned to specifically respond to 
students’ pace and learning styles. 

For this reason, it was very important to trace mathematics teachers’ 
beliefs and misconceptions about assessment in the classroom and to 
analyze learning activities in the classroom focusing on teachers’ rationales 
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behind learning difficulties in mathematics, in order to design and plan 
adequate interventions for remedial programming. 

At the same time, we think that each teacher should acquire the 
necessary competences for reflecting on the content of mathematical 
education as well as on the pedagogical-didactical aspects. This means an 
internal focus on mathematics itself and an external focus on the links of 
mathematics to other subjects. In both cases, it is important for teachers to 
learn, assess and diagnose students’ learning needs and to give answers to 
them based on adequate teaching methodologies. 

During all project activities and implementations, mathematics has 
been seen and validated as a multi-functional and multi-disciplinary 
subject at school. This is based on the fundamental principles that teachers 
need to be aware of their crucial position in school and their need to 
reflect on difficulties and mistakes, to trace their reasons and to plan the 
interventions for remedial programming, through efficacy strategies and 
tools of formative assessment. 

In this perspective it was therefore necessary to train mathematic 
teachers not only in mathematics and didactics of mathematics, but also 
on the application of these disciplines in their own context. In this way, the 
solution lies in designing and giving instruments of educational planning 
and assessment to teachers and teacher trainers. 


3. The partners of the project 


Our project proposal was prepared capitalizing on some previous 
research experiences (and their achieved outcomes) conducted in many 
different countries and concerning the assessment of basic skills in the 
“junior high school’, particularly in mathematics. Indeed, over the past 10 
years, each partner of the project has conducting researches that allowed 
highlighting the important need for training mathematics teachers in the 
field of formative assessment. 

The five Partner Members are all European University Institutions and 
have overall competence in pedagogy and didactics and mathematics. 

In detail, the partners and respective research manager are the 
following: 

e for Italy, Alma Mater Studiorum Universita di Bologna (UNIBO), 
leader of the project with the Departments of Education Studies (Ira 
Vannini and Stefania Lovece) and Mathematics (Giorgio Bolondi, 
Federica Ferretti and Alessandro Gimigliano); 

e for Cyprus University of Cyprus, UCY (Athanasios Gagatsis, Paraskevi 
Michael-Chrysanthou and Theodora Christodoulou); 
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e for Switzerland, University of Applied Sciences and Arts of Southern 
Switzerland — Department of Formation and Learning, SUPSI-DFA 
(Silvia Sbaragli and Miriam Salvisberg); 

e for France, University of Cergy-Pontoise: UCP (Laurent Jeannin and 
Fathia Lograda); 

e for Hogeschool Inholland/Netherlands’ Hogeschool Inholland of Applied 
Sciences, Inholland (Rob Velder). 

Each partner has contributed essentially and decisively to the project, 
providing their experience in various fields. 

Members of Alma Mater Studiorum Universita di Bologna were the 
leaders of the project. The Italian team consists of researchers, assistant 
and full professors of Mathematics, Mathematics Education and Education. 
The research team then provided the essential support at all phases of 
the project; especially in the construction of the shared definition of 
formative assessment in mathematics and of the FAMT&L grid of lesson 
analysis and in the design and implementation of the pilot courses. The 
research group, has been involved for many years in the professional 
development program of teachers at all levels of school both in didactics 
of mathematics in education and this experience has played a crucial role 
for the development of the project. The Alma Mater Studiorum Universita 
di Bologna has also dealt with management; the team includes also present 
administrative and project managers of European projects and technicians, 
experts in usability and ergonomics in the web field. 

The research team of the University of Cyprus is formed by professors 
of mathematics education, experienced researchers and PhD students 
in mathematics education. These experts have a strong experience and 
contributed significantly in data analysis, especially quantitative, using 
different statistical packages (ex. SPSS, Quest, EQS, Excel). Thus, the 
UCY team contributed to the analysis report, which includes the results of 
the survey on teachers’ and students’ beliefs and some directions for the 
pilot training courses aiming to improve the beliefs that emerged in the 
survey. Within the FAMT&L project the role of UCY was — apart from the 
tasks that are filled-in by all partners — focused on the educational/learning 
needs analysis, specifically towards the analysis of teachers’ conceptions 
about teaching and assessment in mathematics. There was also a major role 
of UCY in the planning of the training pilot courses, together with SUPSI- 
DFA as WP co-leader. UCY provided also an important contribution for 
the realization of the project in its whole and, specifically in the process of 
dissemination, interacting with stakeholders at every phase of the project to 
the exploitation of the results. 

Members of Netherlands’ Hogeschool Inholland of Applied Sciences 
are researchers, project manager and ambassador; they are all experts 
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in Lifelong Learning and in Validation of Prior Learning Outcomes- 
systematics. The research team has been engaged for years in the design 
and in the development of one educational model based on competence 
based (lifelong) learning, implying that all teaching and learning activities 
concentrate on the professional competences to be acquired by students 
and maintained through life by them as professionals. This experience has 
been crucial for the validations and the monitoring of FAMT&L project. 

The DFA — Departments of the University of Applied Sciences of 
Southern Switzerland SUPSI — research team is composed by teachers 
and researchers in mathematics education and in education. The SUPSI 
team provides initial teacher education to pre-service primary education 
teachers, a lot of pre-service secondary education teachers and continuing 
education to in-service teachers. All these programs combine theoretical 
modules, professional modules and research activities, in order to allow the 
development of scientific and professional skills. Applied research focuses 
in particular on the understanding and innovation of school systems, on 
teacher education and on education in general. This includes a clear focus 
on mathematics and science teaching and learning, which is researched 
by a pool of experienced researchers and teachers. This research team’s 
experience played a crucial role in the design and implementation of the 
pilot course. 

The University Institute of teachers training of Cergy-Pontoise University 
(UCP) was the biggest institute of France before the modifications of 
training programs. The institute is specialized in the education themes 
and have an important expertise in the development of the teaching and 
learning in distance Education and IT-Competences for teacher and learner. 
All UCP members who have collaborated on the project are experts in 
Educational/learning needs analysis, specifically towards the video analysis 
of mathematics teachers’ practices. The contribution of the researchers’ 
group was fundamental to the design and production of video and its video 
analysis. 

Finally, a fundamental role is that they have associated schools at all 
different levels for collaboration within the project. 

Each country has several “project-associated” schools with which the 
researchers cooperated during together during all phases of the project. 
Other schools were involved as well, although “not officially “associated”, 
mostly in field of dissemination and exploitation. Teachers and students 
of the associated schools have played a key role in the different phases of 
the project. At the beginning of the project, both teachers and students, 
completed questionnaires that were used to outline the schools’ educational 
needs regarding formative assessment in mathematics; some of them 
have also participated in interviews and the collaboration with them has 


43 


Copyright © 2018 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788891774637 


been crucial to deeply understand their needs within the school reality. 
Subsequently, research-training meetings were held where researchers and 
teachers discussed the results of the questionnaires, in order to understand 
and interpret them in a collaborative way. The researchers have thus 
led some professional development meeting about formative assessment 
in mathematics and in video-analysis tools and practices. Along with 
teachers, some key concepts that were then subject to video were outlined; 
partner staff went to the schools to produce videos of real classroom 
formative assessment situation in mathematics. Finally, in most of the 
partner countries, the teachers who participated in the various phases of 
the project were also participants of the pilot course that followed. The 
close and continuous collaboration with teachers and schools was crucial 
not only for teachers, but also for the development of the project. 


4. The aims of FAMT&L Project 


The Project aims to provide a methodological model and some valid 
materials for a quality proposal of mathematics teacher training in Europe, 
starting from an in-depth analysis of training teachers’ needs and on-going 
experiences in the different partner countries. Subsequently, the project 
aimed to promote in-service training of mathematics teachers through the 
design and implementation of an online repository: an innovative product 
that we are using even now that the project is finished, as online resource 
for training activities addressed to mathematics teachers in the schools 
involved, and implementing each training teachers’ national system. In 
particular, the FAMT&L training model has focused on changing the 
students’ and teachers’ beliefs on formative assessment of mathematics. 

In particular, formative assessment, as a strategy to individualize 
teaching strategies and bring all students to achieve positive learning 
outcomes, is a key topic to affect change on mathematics teachers’ 
“culture”. 

One of the major aims of our activities it is to be able to show the 
importance of formative assessment in order to make explicit to teacher 
that assessment: 

e is not an instrument of power in their hands; 

e cannot be used as extrinsic motivation for learning mathematics; 

e is the key to make mathematics teaching adapted to the specific needs 
and students’ learning styles. 

Recapping, the FAMT&L project proposed an innovative path that, 
starting from an investigation of misconceptions and “bad” practices 
of mathematics teachers, got to design a virtual environment (a web 
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repository) for in-service teachers’ training. This learning environment 
provides a variety of tools and objects (examples of learning contexts, 
video of situations of teaching mathematics, assessment tools, training’ 
paths and their specific use in the teaching of mathematics), including a 
guideline to be used by in-service mathematics teachers that participate in 
training courses for secondary schools. 

The main objective of this training methodology was to improve teachers’ 
skills on the use of formative assessment in mathematics education in order 
to promote effective learning for all students. 


5. The Phases of the Project 


The methodological approach of FAMT&L project was based 
on achieving the above-mentioned objectives. It is focused on mutual 
decision making and regular exchange of information in order to ensure 
that all consortium members are involved in the network’s progress. All 
partners committed themselves to share their knowledge and experience 
with the consortium, ranging from longstanding experience in: teaching 
and learning processes concerning mathematics, definition of training 
methodologies and strategies and assessment processes. 

The project phases were the following: 

1. The first part of the project has been dedicated to the analysis of 
teachers’ learning needs through some specific qualitative and 
quantitative research methods (observations, interviews, questionnaires, 
survey, etc.). 

2. In the second phase, the data collected from teachers and students 
were analyzed and the information that derived allowed to design an 
effective training model for teachers of mathematics. 

3. In the third phase, the observation FAMT&L Grid for video analysis 
was designed and realized. All the videos were analyzed using the grid 
and have been then included into the web repository. 

4. The fourth phase consisted of the design and development of the 
training model, realized as an action-training research, where teachers 
will be actively involved and trained to mathematics teaching and 
assessing competences as well as transversal competences such as: 

a) reflexive practice; 

b) self-assessment; 

c) planning and reporting methods; 

d) professional empowerment. 

The support of video analysis tools and techniques was crucial for the 
effective implementation of the teachers training course. 
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During the various training courses implemented by each partner, 
questionnaires were provided to investigate the change in the teachers’ 
beliefs and conceptions on formative assessment, and as described in 
chapter 6, there were actually positive results in this domain. 

According to the planning of the project, the work was distributed to 
partners in Work Packages WP, as follows: 

e Management (WPI); 

e Elaboration of the Report on “teachers’ and student’s beliefs” about 
formative assessment in mathematics and hypothesis about teachers’ 
learning needs (WP2); 

e Elaboration of the Report on practices (or “bad practices”) of the 
mathematics teachers and hypothesis about teachers’ learning needs 
(WP3); 

e Definition of the training model and realization of the pilot training 
courses (WP4); 

e Design and implementation of the web repository for teacher training in 
mathematics (WP5); 

e Analysis of the implementation of the pilot training courses in the 
associated schools partners (WP6); 

e Dissemination and exploitation (WP7 and WP8). 

Different partners were responsible of the processes aimed at reaching 
these milestones, under the overall guidance of the coordinator (UNIBO). 
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4. Assessment in classroom: 
teachers’ and students’ beliefs 


by Paraskevi Michael-Chrysanthou*, Theodora Christodoulou’, 
Athanasios Gagatsis* 


1. Introduction 


In this chapter, the teachers’ and students’ beliefs about the purpose of 
formative assessment in mathematics in the five countries under study are 
analyzed and compared. Our focus is to study how students and teachers in 
each country perceive the purpose of formative assessment in mathematics 
and to explore whether the views between students and teachers resemble 
or differ between the five countries. 

A questionnaire focused on the beliefs about formative assessment and 
assessment in mathematics in general was administrated to all participants. 
The participants were (Tab. 1) 308 Cypriot students, 460 Italian students, 
340 Swiss students, 413 Dutch students and 128 French students filled the 
questionnaire. Respectively, 65 Cypriot teachers, 58 Italian teachers, 69 
Swiss teachers, 51 Dutch teachers and 11 French teachers completed the 
questionnaire designed for the teachers. 

In this chapter, we present the results about the students’ and teachers’ 
beliefs about the purpose of formative assessment in mathematics 
in the five countries participated in the project. Initially, a description 
and comparison of the participants’ responses to the statements related 
to the purpose of formative assessment is conducted. Then, the results 
about the relations between these statements are presented, according 
to the hierarchical clustering of variables. For the particular analysis 
the computer software called CHIC was used (Gras, Régnier, Marinica 
& Guillet, 2013). This method of analysis determines the hierarchical 
similarity connections between the variables. The hierarchical clustering 
of variables is a classification method which aims to identify in a set V of 


* University of Cyprus. 
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variables, sections of V, less and less subtle, established in an ascending 
manner. These sections are represented in a hierarchically constructed 
diagram using a similarity statistical criterion among the variables. The 
similarity stems from the intersection of the set V of variables with a 
set E of subjects (or objects). This kind of analysis allows the researcher 
to study and interpret clusters of variables in terms of typology and 
decreasing resemblance. The clusters are established in particular levels 
of the diagram and can be compared with others. This aggregation may 
be attributed to the conceptual character of every group of variables. In 
this study the similarity diagrams allowed for the arrangement of the 
statements into groups according to their homogeneity. 


Tab. 1 - Participants of each Country 


Students Cyprus Italy Switzerland Netherlands France 
(N=1649) (N=308) (N=460) (N=340) (N=413) (N=128) 
Grade 1 108 247 72 43 17 
Grade 2 72 139 67 152 63 
Grade 3 428 74 78 113 27 
49 

Grade 4 17 

— — 57 105 21 
Teachers 
(N=201) 65 58 69 51 11 


2. The questionnaires 


A questionnaire for examining the teachers’ beliefs regarding the purpose 
and use of formative assessment in mathematics was developed, based on an 
extensive literature review in mathematics education in relation to formative 
assessment and teachers’ educational beliefs (Michael-Chrysanthou, 
Gagatsis & Vannini, 2014). 

The questionnaire comprises of six parts. In the first part (Part A) the 
participants’ demographics are asked. This part includes questions mainly 
about the participants’ gender, age, education and teaching experience. 
In each of the rest five parts the participants mostly have to express their 
agreement or disagreement to different statements on a 4-point Likert scale 
(1=strongly disagree, 4=strongly agree). Negative statements are used as 
well, in order to increase the validity of the questionnaire. Specifically, 
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Part B includes 10 statements examining the first research axis, which 
is about the purpose of formative assessment. In Part C there are 21 
statements about the use of different formative assessment techniques. In 
the next part (Part D) 7 statements are found, examining the participants’ 
beliefs regarding the use of the results of formative assessment. Part E 
comprises of 12 statements for tracing the teachers’ beliefs concerning 
mathematical errors. The last part (Part F) includes 16 issues of assessment 
on which the teachers’ would like or not to be further trained on. 

A questionnaire for examining students’ beliefs for formative 
assessment was also developed (Michael-Chrysanthou & Gagatsis, 
2015). Based on our literature review, various authors’ opinions and 
research results were transformed to statements to be included in our 
questionnaire. Previous relevant research instruments were also traced, 
parts of which were taken as examples for forming some of our statements. 
The questionnaire comprises of two parts. In the first part (Part A) the 
participants’ demographics (gender, age-class and school) are asked. 
Part B includes 44 statements for which students had to express their 
agreement or disagreement on a 4-point Likert scale (1=strongly disagree, 
4=strongly agree). In fact, these statements reflected not only beliefs about 
formative assessment, but also about particular assessment practices used 
by the teacher or/and the students. This structure allows not only tracing 
the students’ beliefs about formative assessment, but also to examine 
the relations between particular practices and the formation of positive 
or negative beliefs. The 44 statements were grouped according to four 
dimensions of formative assessment, as defined based on the results of our 
literature review. In fact, in the first group there were 10 statements about 
the purpose (P) of formative assessment. The second group included 8 
statements about the use of different formative assessment techniques (T). 
In the next group there are 6 statements regarding the use of the results (R) 
of formative assessment, emphasizing on the use of students’ mathematical 
errors. The last group includes 20 statements regarding the role of each 
stakeholder (S) in the formative assessment (students, teachers, parents). 
Representative examples of statements regarding the use of mathematical 
error are provided in the presentation of results. 


3. Teachers’ beliefs about the purpose of assessment in 
Mathematics 


The table summarizes the teachers’ beliefs regarding the purpose 


of formative assessment in the mathematics teaching and learning, as 
reflected by the 10 statements presented. Looking at the total score for 
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each statement, we can claim that the teachers’ beliefs about the purpose of 
formative assessment are generally positive, as the average in the majority 
of the statements is above 3, which shows their (strong) agreement. 


Tab. 2 - Mean of teachers’ responses for the purpose of the assessment in the 
five countries 


Statement 


Variable 


Cyprus 
Italy 
Switserland 
Netherlands 
France 
Total 


Formative assessment 
P1 establishes what students 3.1 
have learned in mathematics. 


N 
œ 
wo 
Li 
wo 
N 
wo 


Formative assessment 

po identifies the students’ 
strong and weak abilities in 
mathematics. 


3.2 | 3.7 | 3.3 | 3.1 3.5 | 3.4 


Formative assessment 
P3 identifies how students think in 3 3.6 3 2.8 3.2 3.1 
mathematics. 


Formative assessment should 
be based on the pupils’ 

Pa outcomes in math rather than 1.9 1.8 Le 

on the process. 


1.7 | 19 | 1.8 


Formative assessment should 

P5 assess the students ability 34 29 
to apply mathematics in 

unfamiliar everyday situations. 


2.5 | 2.6 | 2.8 | 2.8 


The different assessment 
methods aim to assess the 2.9 — 2.4 3.1 2.7 2.8 
students’. 


Knowledge (memorization): 
the ability to memorize rules, 


P6a axioms, theorems and other 3.2 = 3.4 3 3.5 3.3 
mathematical information. 
Comprehension 
(understanding): the ability 

pep |: Perceive mathematical 32 = 36 | 31 | 35 | 34 


meaning and to transform 
mathematical ideas from one 
form to another. 


Analysis: the ability to analyze 
P6c information and to arrive to 3.1 - 3.3 3.1 3.2 3.2 
mathematical conclusions. 
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Tab. 2 - continued 


ke] 2) 
x) a S E Q = 
2 D | >| + | £ 2 S 
£ Statement g g È F S © 
© ni = 5 ra F 
> 2 > | © | 
7) z 
Synthesis: the ability to 
Ped | organize mathematical ideas 32 2 33 | 32 3 3.2 


altogether to form a complete 
image that has meaning. 


The purpose of formative 
assessment is to help 
students improve themselves 
in mathematics. 


P7 2.5 | 3.5 | 2.1 2.1 2.2 | 2.5 


Formative assessment is 
P8 subjective while summative 3.3 2.5 3.3 3 3.5 3.1 
assessment is objective. 


According to the formative 
assessment results, | 

P9 modify my instructional plan 3.1 3.5 | 3.1 2.9 | 3.3 | 3.2 
according to my students’ 
needs. 


Assessing my students’ is very 
useful for me, because it gives 
me a chance to verify the 
validity of my work. 


Total 3 3.1 2.9 | 2.9 3 


P10 2.9 | 3.7 | 2.8 | 31 3.2 | 3.1 


Looking at the total score for country, we also observe that the level 
of agreement in the different statements is slightly differentiated between 
the teachers from the five countries. In Cyprus, Switzerland and the 
Netherlands, teachers express their agreement in most of the statements, 
but not as strongly as the teachers from Italy and France for example, for 
whom we observe a stronger agreement in many of the statements. In fact 
the statements can be grouped into four categories. 

In the first category there are six statements (P1 to P5 and P7) focusing 
on the role of formative assessment in the students’ learning abilities 
and outcomes. Teachers from all countries mostly agree with the role 
of formative assessment in establishing what students have learned in 
mathematics, in identifying the students’ strong and weak abilities in 
mathematics and the way students think in mathematics. This is why they 
express their disagreement to the fact that formative assessment should 
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be based on the pupils’ outcomes in math rather than on the process. 
Thus, the teachers seem to realize that formative assessment focuses on 
identifying the students’ reasoning and abilities, rather than measuring 
their performance. However, teachers’ responses in the statement that the 
purpose of formative assessment is to help students improve themselves 
in mathematics are found in the middle of the scale. This result is in 
contradiction to the previous observation, which could be interpreted by 
the generality of the aforementioned statement, which probably inhibited 
teachers from expressing a strongly positive answer. Furthermore, the 
teachers rather agree that formative assessment should assess the students’ 
ability to apply mathematics in unfamiliar everyday situations. 

The second category reflects beliefs about assessing specific aspects 
of students’ mathematical thinking (P6: a to d), such as Knowledge, 
Comprehension, Analysis and Synthesis. The teachers generally express 
positive beliefs in assessing these aspects of mathematical thinking, with 
the aspect of “Comprehension” to be the most important for them. 

Two statements about the role of formative assessment in relation to the 
teachers (P9 and P10) form the third category. The teachers’ agreement 
to these statements indicate that they recognize that formative assessment 
can have a positive impact on their planning and implementations, as it 
can work as a source for providing them feedback about the effectiveness 
of their teaching and a source of evidence about necessary future 
modifications for adjusting their teaching to their students’ needs. 

Finally, the last category includes the statement P8, which reflects 
the teachers’ beliefs about the nature of formative assessment. Generally 
the teachers agree that formative assessment is subjective, compared to 
summative assessment, which is more objective. However, the power of 
this belief is not the same among the teachers from the five countries. 
Actually, the strongest agreement is observed in the French teachers, 
whereas the weakest is found in the Italian teachers. 

In the similarity diagram for the Cypriot teachers’ beliefs on the 
purpose of formative assessment (Fig. 1) three distinct similarity clusters 
are distinguished. In the first similarity cluster there are two statements 
referring to the students (S1 and S4), one statement about the teachers 
(P10), a statement about the nature of formative assessment (P8) and a 
statement about a specific aspect of students’ mathematical thinking (P6a). 
In the first subgroup in this cluster the relation between the variables PI, 
P10 and P6d show that the teachers who believe that formative assessment 
establishes what students have learned in mathematics believe also that 
formative assessment is a source for verifying the validity of their work, 
and thus assessing students’ knowledge (memorization) is important for 
them. 
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Fig. 1 - Similarity diagram of the Cypriot teachers’ beliefs about the purpose of 
formative assessment 


In the next cluster we observe coherence in the teachers’ beliefs 
about formative assessment in relation to the students (P2, P3 and 
P7). This is why there are three statements about the students, which 
are related to a statement about the teachers (T9). The teachers have 
related beliefs about the benefits of formative assessment for the 
students’ thinking, identification of their abilities and improvement in 
mathematics. These beliefs influence also their belief that their teaching 
should be modified in order to be adjusted to the students’ needs. Thus, 
this cluster reveals that the Cypriot teachers seem to realize the positive 
effect of formative assessment on the students’ mathematical thinking 
and that formative assessment can help them meet their students’ needs 
and plan their next lesson in order to help their students as much as they 
need. 

Four variables form the last similarity cluster of this diagram. One 
refers to the students (P5), whereas the rest three refer to specific aspects 
of students’ mathematical thinking (P6b, P6c and P6d). Actually the 
teachers that consider important to assess the students’ ability to apply 
mathematics in unfamiliar everyday situations, consider also essential to 
assess the students’ comprehension, analysis and synthesis abilities. These 
dimensions of mathematical thinking are really important and directly 
related to applying mathematics in our lives. 

The similarity diagram of the Italian teachers’ beliefs (Fig. 2) is formed 
by three similarity clusters. In the first cluster we can distinguish two 
subgroups of variables. 

In the first subgroup three statements regarding the purpose of 
formative assessment in relation to the students’ learning are grouped 
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Fig. 2 - Similarity diagram of the Italian teachers’ beliefs about the purpose of 
formative assessment 


(P1, P4 and PS). In fact, a significant and strong relation is formed 
between the statements P1 and P4, indicating that the Italian teachers’ 
belief that formative assessment establishes what students have learned 
in mathematics is strongly related to the belief that formative assessment 
should be based on the pupils’ outcomes in math rather than on the 
process. In the second subgroup in this cluster we observe a relation 
between a statement that is related to the teachers (P10) and one statement 
related to a particular aspect of students’ mathematical thinking (P6d). 
Particularly, the teachers’ belief that assessing their students is a source 
for verifying the validity of their work is related to their belief that it is 
important to assess the students’ Synthesis abilities. This is an indication 
that the Italian teachers focus a lot on the students’ Synthesis abilities in 
order to verify the effectiveness of their teaching. 

In the second similarity cluster four variables are significantly grouped. 
Two of these variables reflect statements about the purpose of formative 
assessment in relation to the students (P2 and P3), a statement about the 
nature of formative assessment (P9) and a statement about particular aspect 
of students’ mathematical thinking (P6a). In fact the statements P2 and 
P6a form a first subgroup in this cluster, showing that the teachers’ belief 
that formative assessment identifies the students’ strong and weak abilities 
in mathematics is affected by the belief that it is important to assess 
the students’ knowledge (memorization). Thus, for the Italian teachers 
assessing students’ abilities to memorize rules, axioms, theorems and 
other mathematical information is important for tracing their strong points 
and weaknesses. The second subgroup in this cluster is formed by the 
statements P3 and P8. Therefore, the Italian teachers’ belief that formative 
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assessment identifies how students think in mathematics affects their belief 
about the nature of formative assessment. 

In the last similarity cluster two relations between four variables are 
formed. In fact, two statements about particular aspects of students’ 
mathematical thinking (P6b and P6c), one statement about the teachers 
(P9) and one statement about the students (P7) are found. In the first 
relation the teachers’ belief that the purpose of formative assessment is 
to help students improve themselves in mathematics is related to their 
belief about the importance of assessing the students’ Comprehension 
(understanding). Thereafter, the Italian teachers appear the students’ 
ability to perceive mathematical meaning and to transform mathematical 
ideas from one form to another as an aspect that shows their progress 
in mathematics. The next relation is about the belief that formative 
assessment can be used for helping the teachers modify their instructional 
plan according to the students’ needs and the importance of assessing the 
students’ Analysis abilities. Consequently, the Italian teachers consider 
important gathering evidence about the students’ ability to analyze 
information and to arrive to mathematical conclusions for defining their 
students’ needs and planning their future lessons according to them. 

In the similarity diagram about the Swiss teachers’ beliefs on the purpose 
of formative assessment three distinct similarity clusters are identified. In 
the first similarity cluster five variables are grouped, with three of them 
referring to assessing particular aspects of the students’ mathematical 
thinking (P6a, P6c and P6d) and two of them referring to the students (P1 
and P4). Specifically, students’ Knowledge (memorization), Analysis and 
Synthesis abilities are important to be assessed for the Swiss teachers, in 
order to be able to establish what students have learned in mathematics and 
to focus on the pupils’ outcomes in math rather than on the process. 


so o 3 o 
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Fig. 3 - Similarity diagram of the Swiss teachers’ beliefs about the purpose of 
formative assessment 
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In the next similarity cluster most of the statements regarding the 
purpose of formative assessment in relation to the students (P2, P3 and 
P5) are found together with the statement about the nature of formative 
assessment (P8). This cluster indicates the coherence in the Swiss teachers’ 
beliefs about the effect of formative assessment in the students’ learning 
and thinking. These teachers’ beliefs that formative assessment aims at 
identifying the students’ strong and weak abilities in mathematics, the 
way they think in mathematics and their ability to apply mathematics in 
unfamiliar everyday situations are coherent and influence their belief about 
the nature of formative assessment. 

The last similarity cluster includes two relations between four 
variables. The first relation is formed between the belief that the purpose 
of formative assessment is to help students improve themselves in 
mathematics (P7) and about the importance of assessing the students’ 
comprehension (understanding) (P6b). Therefore, the Swiss teachers 
consider the assessment of students’ comprehension (understanding) as a 
way to help them get better in mathematics. The next relation is created 
between two statements about the purpose of formative assessment in 
relation to the teachers (P9 and P10). In fact, this is a strong and important 
relation, indicating coherence in this kind of beliefs. Actually, teachers 
recognize the important role of formative assessment in verifying the 
validity of their work and in modifying their next steps in relation to the 
students’ needs. 

The Dutch teachers’ beliefs regarding the purpose of formative 
assessment are organized in three distinct similarity clusters, as shown in 
the Figure 4. 


Fig. 4 - Similarity diagram of the Dutch teachers’ beliefs about the purpose of 
formative assessment 
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In the first similarity cluster two subgroups of variables are 
distinguished. The first one is formed by the relation between two 
statements about the students (Pl and P7) and one statement about a 
specific aspect of mathematical thinking that should be assessed (P6a). 
This relation indicates that the teachers’ beliefs that formative assessment 
has the potential to help students improve themselves in mathematics 
and that also establishes what students have learned in mathematics are 
interrelated. Those teachers’ beliefs are related to the belief that students’ 
Knowledge (memorization) is an important aspect of mathematical 
thinking to be assessed, as it is probably an indication for them about 
the students’ learning and improvement. The second subgroup is formed 
by a significant relation between a variable about the students (P4) and a 
variable about the nature of formative assessment (P8). In fact, this relation 
shows that the teachers relate the fact that formative assessment should 
be based on the pupils’ process in math rather than on the outcomes with 
that formative assessment is subjective while summative assessment is 
objective. Both those statements reflect the nature of formative assessment 
and thus this is probably why they are significantly related. 

In the next similarity cluster four variables are related, two referring to 
the students (P2 and P3) and two referring to the teachers (P9 and P10). 
The first relation in this cluster reveals that the teachers realize the benefits 
of formative assessment in identifying the students’ strong and weak 
abilities in mathematics (P2) and that this is important for them in order 
to modify their instructional plans according to their students’ needs (P9). 
The second relation shows a similar realization, as the teachers’ belief that 
formative assessment identifies how students think in mathematics (P3) is 
related to their belief that assessing their students is very useful for them as 
it help them verify the validity of their work (P10). 

The last similarity cluster is formed by the significant relation between 
four variables. Three of them refer to particular aspects of the students’ 
mathematical thinking (P6b, P6c and P6d), whereas the other refers to 
the students (P5). In particular, when the teachers believe that formative 
assessment should assess the students’ ability to apply mathematics 
in unfamiliar everyday situations, they also consider important to assess 
students’ Comprehension (understanding) and Analysis and Synthesis 
abilities. Thus, those dimensions of mathematical thinking are discriminated 
by the teachers as the most necessary for using mathematics in everyday life. 

The relations between the French teachers’ beliefs are illustrated in 
Figure 5. In this diagram the teachers’ beliefs are organized into two 
similarity clusters, which are for the first time related significantly. Thus, 
this is the first time that coherence appears between the teachers’ beliefs 
regarding the purpose of formative assessment. 
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In the first similarity cluster two subgroups are observed. In the first 
one five variables are significantly related. Specifically, one variable related 
to the students (P1) is related to four variables regarding specific aspects 
of students’ mathematical thinking (P6a, P6b, P6c and P6d). In fact, 
this relation reveals that when teachers believe that formative assessment 
establishes what students have learned, they consider important to assess 
the students Knowledge (memorization), Comprehension (understanding), 
Analysis and Synthesis abilities. The second subgroup consists of a relation 
between a variable related to the students (P4) and a variable about the 
nature of formative assessment (P8). 

In fact, when the teachers consider that formative assessment 
should be based on the pupils’ process in math rather than on the 
outcomes they also believe that formative assessment is subjective 
while summative assessment is objective. Actually, both statements 
refer to the nature of formative assessment and this can explain their 
significant relation. 


|_| 


Fig. 5 - Similarity diagram of the French teachers’ beliefs about the purpose of 
formative assessment 


Similar to the first similarity cluster, the second one consists also 
of two subgroups. In the first subgroup, a relation is formed between 
two variables about the students (P2 and P7) and a variable about the 
teachers (P9). These relations indicate that for the teachers who believe 
that formative assessment can contribute in identifying the students’ 
strong and weak abilities in mathematics (P2), formative assessment is a 


58 


Copyright © 2018 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788891774637 


source for providing them evidence in order to modify their instructional 
plans for satisfying their students’ needs (P9). In relation to these, they 
also believe that among the purposes of formative assessment is to 
help students improve themselves in mathematics. Thus, the teachers 
recognize that the benefits of formative assessment are affected by the 
teacher-students interaction. The second subgroup includes the relation 
between two variables about the students (P3 and P5) and a variable 
about the teachers (P10). Actually, this relations show that the teachers’ 
belief that formative assessment identifies how students think in 
mathematics (P3) is related to their belief that assessing their students 
is very useful for them as it help them verify the validity of their 
work (P10) and that formative assessment should assess the students’ 
ability to apply mathematics in unfamiliar everyday situations (P5). 
Thereafter, this relation reveals once again the teachers’ recognition that 
formative assessment can have positive effects both on the teachers and 
the students and that the interaction between them is an important aspect 
for the effective use of formative assessment. 


4. Students’ beliefs about the purpose of assessment in 
Mathematics 


The following table (Tab. 3) summarizes the students’ beliefs about 
the purpose of assessment in the mathematics teaching and learning, as 
reflected by the 10 statements presented. Looking at the total score for 
each statement, we can claim that the students’ beliefs about the purpose 
of formative assessment are neutral, as the mean in the majority of the 
statements is between 2 and 3, which shows a relative agreement in the 
statements, but not strong. Looking at the total score for each country, 
we also observe that the level of agreement in the different statements 
is slightly differentiated between them. In Netherlands and France the 
students express less agreement in most of the statements, compared to 
the rest of the countries. In fact the statements can be grouped into two 
categories, according to the domain they refer to. 

In particular, the first category consists of five statements (PI to P5) 
focusing on the purpose of the formative assessment in the students’ 
cognitive domain and in their understanding. Students from all countries 
mostly agree with the role of formative assessment in the identification 
of their good skills in mathematics. This statement (P1) has the highest 
mean, which shows that the students’ realize more explicitly this purpose 
of assessment. The rest four statements included in the first category 
present a lower mean. This indicates that most of the students from all the 
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Tab. 3 - Mean of students’ responses for the purpose of the assessment in the 
five countries 


Le) n 
T 
© o Ei E o | _ 
2 D > = s © El 
© Statements © © © 3 E 5 
oO (©) = =] LL 
> > © 
” z 
Assessment helps me 
P1 identifying my good skills in 2.8 2.9 | 29 | 26 | 2.8 


math. 


Assessment does not help 
P2 me facing my difficulties on a 2.8 | 2.9 2.1 2.5 | 2.6 | 2.6 
mathematical subject. 


The grades that | receive 
on a math test cannot show 
P3 if | have understood the 2.6 2.2 2.5 2.5 2.6 2.5 
mathematical subjects | have 
been taught. 


Some assessments serve 
to verify only what | have 
P4 understood on a mathematical | 2.5 | 2.8 | 2.7 1.8 | 2.6 2.5 
subject and not for our grade 
report. 


When feedback is continuous 
| feel | have a foundation that 


P5 helps me to understand what | ae 2.6 2.5 2.5 26 
am learning in math. 
P6 Assessment in math provokes 22 | 27 | 25 21 25 | 24 


me anxiety. 


| feel more confidence about 
myself when | have more 

P7 frequent feedback about my 2.9 | 2.9 | 2.7 | 2.3 | 2.4 | 2.6 
progress in a mathematic 
subject. 


Assessment information 
P8 motivates me to set new goals | 2.6 | 3.2 3 2.3 | 2.6 | 2.7 
in learning math. 


When | am not satisfied about 
the grades that | have received 
for my working in math, | have 
to try harder. 


P9 3.2 | 3.4 | 3.2 | 2.9 | 2.4 3 


The grades and the reports in 
P10 math do not force me to work 2.7 2.1 2.6 2 2.4 2.4 
when | don’t want to do. 


Total 2.7 | 2.8 | 2.7 | 2.4 | 2.5 
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countries agree less with these statements and thus their beliefs about 
them are more neutral. It is noteworthy that the students’ beliefs about 
the statement P4 are very different between Netherlands and the rest 
countries under study. In particular, the Swiss students disagree with this 
statement, while most of the students in the other countries agree that 
“some assessments serve to verify only what they have understood on a 
mathematical subject and not for our grade report’. Generally, most of the 
students seem to have positive beliefs regarding the role of assessment in 
their cognitive domain, because the average in all the statements is higher 
than two. 

The second category includes the statements P6 to P10, which refer 
to the role of assessment in the students’ affective domain (e.g. anxiety, 
satisfaction, motivation). At a first glance, we observe that students in 
Cyprus, Italy and Swiss are more positive than those in Netherlands and 
France in these five statements. In general, most of the statements indicate 
the neutral beliefs of the students in all countries. A differentiation appears 
in the statement P9, which presents the biggest agreement by the students 
from all countries except France. The results of the statements in this 
category bring us to the conclusion that assessment does not have such a 
large impact on the students’ affective domain. 

In the first similarity diagram (Fig. 6) about the Cypriot students’ 
beliefs, two similarity groups are formed, which are not connected. This is 
an indication that the Cypriot students do not have a coherent view about 
the purpose of assessment in mathematics. 
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| | | 


Fig. 6 - Similarity Diagram about the Cypriot Students’ Beliefs about the Purpose 
of the Assessment in Mathematics 


The first similarity group comprises of six variables that are divided 
into two subgroups. The first subgroup includes the variables P1, P5 and P7, 
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which are significantly related. Among these three variables, the strongest 
and significant similarity relationship is identified between the variables 
P1 and P5, which refers to the students’ cognitive domain. In particular, 
they refer to the possibility that assessment provides to the students for 
identifying their strengths and verifying their understanding in mathematics. 
These two variables are linked with the variable P7. The second subgroup 
consists of three variables (P4, P8, P9), which are connected because they 
refer to the students’ motives for assessment and their affective domain 
in general. Specifically, the strongest similarity relationship is observed 
between variables P8 and P9, which are related with the students’ motives 
and how these are affected after an assessment. Among all the variables that 
form the first similarity group a weak, but significant similarity relationship 
is observed. This probably occurs because the variables that form this 
similarity group refer to the positive dimension of assessment, especially to 
the students’ cognitive and affective domain. 

The second similarity group includes two pairs of variables, which are 
grouped because they highlight the negative aspect of assessment in both the 
cognitive and the affective domain of the students. In particular, we observe 
that the strongest similarity relationship lies between the variables P2 and 
P6. According to these two variables, the assessment does not help students 
to overcome their difficulties in mathematics and this causes anxiety to 
the students. The variables P3 and P10 are also strongly related and they 
refer to the negative influence of grading on the students’ understanding of 
mathematical concept and their effort to overcome their difficulties. 

In the similarity diagram (Fig. 7) for the Italian students we observe 
that two similarity groups are formed, which are not connected. This is 
an indication that the Italian students do not have a general view about the 
purpose of assessment in mathematics, but they isolate some of its aspects. 


Fig. 7 - Similarity Diagram about the Italian Students’ Beliefs about the Purpose 
of the Assessment in Mathematics 
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Analyzing each similarity group separately, we observe that the 
first similarity group comprises of six variables that are divided into two 
subgroups, which are significantly related. The strongest and significant 
similarity relationship lies in the first subgroup and links the variables P1 and 
P2. The grouping of these variables has probably emerged due to the fact that 
they refer to the students’ strengths and difficulties in mathematics. However, 
the relations in the two pairs of variables that form the second similarity 
subgroup appear less strong but still significant. These variables are connected 
because they are related with the students’ affective domain. In particular, in 
the second subgroup, the strongest similarity relationship is observed between 
the variables P8 and P9, which mainly refers to the students motives and 
how these may be affected by an assessment. The pair of variables P5 and P7 
also forms a strong similarity relation. These variables refer to the students’ 
affective domain, which it seems to affect their learning in mathematics. 

The second similarity group includes two pairs of variables which are 
grouped because they refer to the negative aspect of assessment on the 
students’ cognitive and affective domain. In particular, we observe that the 
strongest similarity relationship in this similarity group is identified in the 
pair of variables P3 and P10, which are related to the negative influence 
of grades on the students’ mathematical understanding and their effort 
to overcome their difficulties. The variables P4 and P6 have a less strong 
relation, but according to them, some assessments may be conducted in 
order to verify the students’ understanding and not exactly for the purpose 
of the grade in the progress report. However, these kinds of assessment 
cause anxiety to the students. 

In the following similarity diagram (Fig. 8) about the students from 
Swiss two similarity groups are formed, which are not connected. This is 
an indication that the Swiss students’ beliefs are compartmentalized. 
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Fig. 8 - Similarity Diagram about the Swiss Students’ Beliefs about the Purpose 
of the Assessment in Mathematics 
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Analyzing each similarity group separately, we observe that the 
first similarity group consists of six variables that are divided into two 
subgroups. The first subgroup connects the variables P1, P5, P8 and P9, 
which are significantly linked. The variables that form the first similarity 
subgroup refer to the positive dimension of assessment, especially to the 
students’ cognitive and affective domain. In particular, the strongest and 
significant similarity relationship is identified between the variables P8 and 
P9, which are related with the students’ motives and how these are affected 
after an assessment. The pair of the variables P1 and PS also appears with 
a very strong similarity relation between the two variables, which refer to 
the students’ cognitive domain. In particular, they refer to the possibility 
that assessment provides students the possibility to identify their strengths 
and verify their understanding in mathematics. The second similarity 
subgroup consists of the variables P4 and P7, which refer to the students’ 
understanding and their progress in mathematics. In general, among all the 
variables in the first similarity group, we observe a weak, but significant 
similarity relation. This probably occurs because the variables that form 
this similarity group refer to the positive dimension of assessment and 
especially to the students’ cognitive and affective domain. 

The second similarity group is formed by two pairs of variables 
which appear a relatively weak similarity relation, but they are grouped 
because they are related with the negative aspect of assessment, in both the 
cognitive and the affective domain of the students. In particular, we observe 
that the strongest similarity relationship in this similarity group is identified 
in the pair of the variables P3 and P10. These variables are related with 
the negative influence of the grade on the students’ understanding of 
mathematical concept and their effort to overcome their difficulties. The 
similarity relation between the variables P2 and P6 is quite strong and 
according to these variables, the assessment does not help students to 
overcome their difficulties in mathematics, so this stresses them. 

In the next similarity diagram (Fig. 9) for the Dutch students we 
observe that two similarity groups are formed, which are not connected. 
This is an indication that the Dutch students’ beliefs about the purpose of 
assessment in mathematics are not comprehensive. 

The first similarity group comprises of six variables which are divided 
into two subgroups. The first subgroup connects the variables P1, P5, 
P7 and P8, which appear a relatively weak, but significant similarity 
relation. However, the variables that form the first similarity subgroup 
refer to the positive dimension of assessment, especially to the students’ 
cognitive and affective domain. In particular, the strongest and significant 
similarity relationship is observed between the variables P7 and P8. These 
variables are related with the students’ confidence and motives and how 
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Fig. 9 - Similarity Diagram about the Dutch Students’ Beliefs about the Purpose 
of the Assessment in Mathematics 


they are affected by an assessment. The pair of the variables P1 and 
P5 also appears with a strong similarity and it refers to the students’ 
cognitive domain. In particular, they refer to the possibility that assessment 
allow students to identify their strengths and verify their understanding in 
mathematics. The second similarity subgroup consists of the variables P6 
and P9, which appear a strong similarity relation and refer to the impact 
of assessment on the students’ affective domain. In general, among all 
the variables of the first similarity group, a weak but significant similarity 
relationship is observed. However, the similarity relation between these 
variables is very weak so that, the grouping of two subgroups of the 
variables cannot be precisely interpreted. However, we could say that the 
first similarity group is composed of two groups of variables that link the 
two dimensions of assessment; the positive one in the first subset and the 
negative one in the second subgroup. Also, most of the variables in both 
subgroups mainly refer to the students’ affective domain and how it is 
affected (positively or negatively) by assessment. 

The second similarity group consists of two pairs of variables which 
appear very weak similarity relations. However, these variables are 
grouped because they are related with the understanding of mathematical 
concept and the general impact of assessment in students’ cognitive 
domain. In this similarity group, the strongest similarity relationship is 
found in the pair of the variables P4 and P10, which refers to the grade 
of the assessment and its effect on the effort of the students in order to 
overcome their difficulties. The similarity relation between the variables 
P2 and P3 is quite strong and according to these variables, the assessment 
does not help students overcome their difficulties in mathematics, nor gives 
information about students’ understanding for the mathematical concept. 
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In the last similarity diagram (Figure 10) about the French students’ 
beliefs we observe that two similarity groups are formed, which are not 
connected. This is an indication that the French students do not have 
coherent beliefs about the purpose of assessment in mathematics. 
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Fig. 10 - Similarity Diagram about the French Students’ Beliefs about the 
Purpose of the Assessment in Mathematics 


The first similarity group comprises of six variables which are divided 
into three pairs. The strongest and significant similarity relationship links 
the variables P9 and P10, which refer to the students’ affective domain 
and to the grades that result from assessment. Less strong similarity 
relationship is presented by the variables P3 and P4, which are related 
to the students’ understanding of the mathematical content and how the 
assessment helps them verify if they understood the content taught. A 
weaker similarity relation is found between the variables P1 and P7, which 
are not related to each other. A very weak similarity relation is observed 
connecting all the variables of the first similarity group. However, they 
are grouped because most of them refer to the grade and the information 
given by assessment about the students’ understanding of the mathematical 
content taught. 

The second similarity group consists of two pairs of variables which 
are related with a very weak but significant relation. However, they are 
grouped because they represent statements that mostly refer to the impact 
of assessment on the students’ affective domain. In particular, we observe 
that the strongest and significant similarity relationship in this similarity 
group is found in the pair of variables P5 and P6, which refers to the 
students’ affective domain and how it is influenced either positively or 
negatively by the assessment. The variables P2 and P8 form a less strong 
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similarity relation and they refer to the role of assessment in addressing 
students’ difficulties in a mathematical content and setting new goals in 
learning of mathematics. 


5. Conclusions 


The teachers from all the countries have positive beliefs about the 
purpose of formative assessment in their teaching and the students’ 
learning. They realize that formative assessment can have a positive impact 
on their teaching, as it helps them recognize their students’ thinking, 
understanding and level of abilities and thus can use this information for 
adjusting their next lessons in relation to the students’ needs. Students are 
also improved by this process, as their needs, abilities, conceptions and 
misconceptions are identified and proper actions are designed according 
to them. The only point that the teachers’ beliefs are slightly differentiated 
regards the nature of formative assessment, either as subjective or 
objective, in contrast also to the summative assessment. However, they 
seem to consider summative assessment as more objective in relation to 
formative assessment. 

The students from Cyprus, Italy and Switzerland have most positive 
beliefs about the purpose of the assessment both in the cognitive and 
affective domain. The biggest agreement between the students from all 
the countries appears in the view that “when they are not satisfied about 
the grades that they have received for their working in math, they have to 
try harder’. However, in fact, the students’ beliefs from all the countries 
are not clearly positive towards the role of the assessment in mathematics, 
but we could claim that they are mainly neutral. Thus, the impact of the 
assessment in students’ understanding about mathematics or their feelings 
about mathematics is not very large. 

Generally, the students’ conception about assessment possesses some 
formative characteristics, but they do not consider assessment generally as 
formative. On the other hand, they tend to link assessment with grading, 
thus summative assessment is more grounded in the students’ beliefs. 
This can be explained in relation to the students’ learning and teaching 
experiences, which seem to apply assessment in a more summative form. 
However, their teachers seem to give a more formative character to 
assessment, as they agree with the benefits of formative assessment on 
the students’ learning and on the improvement of their own practices. 
This brings us to the conclusion that there is a contradiction between the 
teachers’ beliefs about the characteristics of formative assessment and their 
actual assessment practices in classroom. 
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5. Theoretical framework 
of video-analysis methodology 


by Laurent Jeannin* 


1. Introduction 


In research, there is a question which is not insignificant: collecting 
and using the data. The FAMT&L project has two objectives. The first is 
to have a state of representations of a category of teachers, on the notion 
of formative assessment, in a particular discipline, here mathematics 
at a given level of education: secondary. On the basis of these results, 
it is then proposed to set up a training program for novice teachers 
(beginners) based on video excerpts. Three issues emerge from the two 
above-mentioned objectives, namely, whether the video data in question 
should enable: 

e to compare the «declarative representations» of teachers and their 
actual or non-actual implementation in real situations? 

e to provide training support and thus to present all the characteristics 
necessary for the exercise of training of type autoscopy?! 

e to be both? 

In order to work on this point of view, the project consortium first 
took note of the history of video data in social sciences and humanities 
research and then decided and applied a method of collecting, 
processing, cutting and use in training sequences. This chapter therefore 
aims to take the different stages worked during the project around the 
notion of video data. 


* Université de Cergy-Pontoise, Paris. 
1. Autoscopy is a concept of video training: autoscopy is literally a self-focused look 
through the image. Through autoscopy, we can analyse his professional practice. 
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2. Point of view of history/representations of the uses of 
the videos in researches 


There are two contradictions in research in Arts, Humanities and Social 
Sciences: (1) distrust of the visual and (2) the explosion of technology and 
mass consumption of images, which have consequences: 

e Theoretical and methodological construction of the visual appeal; 

e Data analysis methodology to implement; 

e Archiving, metadata. 

The researchers use different ways of video data, which you can find 
some examples below: 

e Anthropology and Sociology: photo and documentary film; 

e Visual Cultural Studies: Visual Culture, receipt and consumption of 
public images, media; 

e Mass Media studies; 

e Women/Feminist Studies: Critical to the use of images of the female 
body (Cartwright); 

e Studies of social interaction in sociology/linguistics analysis of the 
organization of action, social practices, analysis of the uses of language 
located, Corpus audio from the 60s to today, video; 

e Study gestures (psychology, linguistics, etc.); 

e Ergonomics, workplace studies; study of work situations; 

e Study of Sign Language; 

e Science education, for example: Observation and Research Centre on 
the teaching of mathematics and Michelet Schools Talence: COREM, 
TIMSS, CLASS... 

In summary of this non-exhaustive list of research applications in 
the Arts, Humanities and Social Sciences, we can easily realize that 
the collect of video data and its use do not have the same objectives 
involving different methods: situation called “authentic” “to be a 
reflection of a culture at a specific moment, or to allow the collect 
of a trace of a social or cultural phenomenon before its extinction, 
the researcher who positions himself as the one who films or even 
orchestrates a situation “In the laboratory” to have all the characteristics 
that are necessary. 


2.1. Point of view of Historical References 


e Late nineteenth century, Haddon uses the movie to fix the population 
costume of Strait Toress. 
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Malu-Bomai Ceremony at Kiam (ca. 6 September 1898) 


Three men in forest setting wearing leaf skirts: leading man wears the cardboard mask made for Haddon and last man holds a 
tailpiece. They dance in procession. Length 50 seconds at 16 £p.s 


Fig. 1 - Malu-Bomai Ceremony at Kiam -— Torres Strait Islander Ceremony 
Documentary, Anthropological Footage, Travelogue). Haddon, Alfred Cort. (non- 
indigenous director/anthropologist), 1898 


The researchers did use films between the two wars: eg documentary 
film by Grierson. The visual anthropology explicitly begins with Mr. Mead 
and Bateson in the case of the study of Balinese culture: a film to exceed 
the limits of verbal language and to tell/show “etos the” eg “the intangible 
aspects of culture”. 

In the interdisciplinary project The natural history of an interview 
(1955), there are a multi-disciplinary analysis of a film documenting the 
psychiatric consultation Bateson with Doris patient and a participation of 
Birdwhistell, which is developing a system of transcription and precise 
kinetic annotation video data: 

e Our primary data are the multitudinous details of vocal and bodily 
action recorded on this film. We call our treatment of such data a 
“natural history” because a minimum of theory guided the collection 
of the data. The cameraman inevitably made some selection in his 
Shooting; and “Doris”, the subject of the interview, was selected 
for study not only because she and her husband were willing to be 
studied in this way but also because this family suffered from inter- 
personal difficulties which had led them to seek special psychiatric 
aid. 

e McQuown, 1971, The Natural History of an Interview. University of 
Chicago Library Microfilm Collection of Manuscripts in Cultural 
Anthropology, series 15, Nos, 95-98. 
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The pioneers of the Video are Kendon and Goodwin, Adam Kendon 
who is a pioneer in the study of gestures in a natural situation: movies 
from the 60s and Charles Goodwin who is the global standard for 
multimodal interaction study: movies from the early 70s. 


2.2. Point of view of diversified uses 


The diversified uses of video in research in Humanities and Social 

Sciences are listed in the following non exhaustive list: 
e Dissemination of content: a viewer to read: 
— video results as restoring means or used for communication; 

— cutting by chapter or streaming. 

e Elicitation of speech organized by researchers: 

— video-elicitation (Krebs, 1975 Balinese dance in a quasi-experimental 
perspective Asch 1980 trance in Bali, in a phenomenological 
perspective); 

— self-confrontation in ergonomics (Theureau), clinical psychology 
(Clot). 

e Documentation of events, activities, social practices to an analytical 
reading: 

— active viewing, selection of extracts, navigation in the video; 

— video as material given corpus (Kendon Goodwin). 

e Referential reading versus praxeological reading: 

— referential reading emphasizes content, like video memorial and 

informational support; 

— praxeological reading favors the organization of the action video as 
making available a phenomenon to be analyzed in its temporality 
and dynamism (in game process). fundamental importance of time: 
both the overall time for action and time details of a movement, an 
adjustment to the other. 

In conclusion, first developed in the United States, research on teacher 
practice resulted in many works. They have long been enrolled in a 
paradigm of “process-product” by identifying variable categories (Durand, 
1996; Anderson, 1983; Brophy, 1983; Doyle, 1983, 1986; Crahay, 1989) that 
influence student learning but reducing the study of the teaching process 
only to observable behavior of the teacher. These studies were designed to 
determine the “effectiveness” of education (Walberg & Fowler, 1991) and 
are still present today with a consideration of the “performance” of pupils 
(TIMSS, 1995 and 1999). 

Secondly, researchers have developed the cognitive models “thinking 
of teachers” (Shalvelson, 1981; Tochon, 1993) who studied the cognitive 
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nature of education: preparations, planning and decision making affecting 
practices. 

Thirdly, the “ecological” models have rehabilitated the importance of 
the “situation” (Bronfenbrenner, 1986) or the instructional. Finally, in last 
decade, interactionist and plural models (Robert, 1999; Rogalsky, 1999) 
have developed. They articulate several types of variables: the teacher, the 
learner and the “situation”. 

According to Beillerot (1998) “the practice, although they included the 
idea of the application, do not immediately return to how and gestures, 
but the methods to do. The practice is at once the rule action (technical, 
moral, religious) and the exercise or its implementation. This is the double 
dimension of the concept of practice that makes it valuable: on one hand, 
gestures, behaviors, languages; the other, through the rules, these are the 
objectives, strategies, ideologies which are invoked”. 

In order to treat teachers’ practices, researchers have usually two 
methods: taking open notes (written notes, schemes, drawings) or the 
use of a coding grid, sometimes supplemented by copies of documents 
or a collection of objects created or used (Barron, 2007). The problem 
is the number of constraints that occur during their use: the accuracy 
of the human eye, write speed when taking notes, the necessity of a 
long immersion in a population, reproducibility of data, elaboration of 
observation grids before observation and determination of categories. 


3. Methods for recording video in the classroom 


Since the development of digital technologies and the extension of the 
video in the educational research field, different methodological practices 
to collect and to analyze data from video recordings have emerged. 


3.1. Video recording equipment in classroom settings 


The primary concern before starting to record practices on the 
classroom should be the choice of the video camera(s) and the positioning 
of the camera(s). 

Before questioning the technical protocol, it is necessary to define, 
based on the theoretical framework, the types of data required. For 
example, access the contents of knowledge exchange between students 
need to have a quality recording, but whose? a student dyad, but then how 
to choose? how will they be representative of the entire class, an entire 
age? to have simultaneously the teacher, class and several pairs or groups 
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of students? Each of the protocols we have just stated affects the sound 

acquisition device and therefore the effect of the record. 

Recording a group of students need to have a strong and clear sound for 
each student in the group, so at least two devices are possible: 

e an area microphone in the group center, on a table in height (10 to 15 
cm) to avoid taking the sounds of objects against the table and good 
distances mouths. One micro manage, but also risks to not hear a 
student who moves too far or not actually in front of the microphone; 

e a wireless microphone per student and thus we have a high-quality sound 
reproduction for every student. Depending on the number of students 
in the group, it can be complicated to manage multiple frequencies, 
multiple batteries for the microphones, multiple receivers, a mixer to 
combine all sources on a single recording track. 

Regarding the teacher, two methods are possible: 

e with a unidirectional microphone gun manipulated by an external 
operator to the teaching sequence. A sound of a high quality without 
battery problem, but someone outside the class that always drives a 
shotgun microphone in the direction of the teacher; 

e with a wireless microphone, without the pressure of the shotgun 
microphone but with risks of tapping the microphone to have a battery 
concern, or a reception problem between the microphone and the 
receiver. 

Then, as the analysis will focus on the articulation between the various 
resources: teacher talk, room ambience, issue of students in class, groups... 
so it will be necessary to have synchronization all sound sources. 

Two devices are also possible: 

e Like in the movies, with a “clap” start, which is present on each sound 
recording and then allows it to declare clap as the zero time on what is 
called the time code? 

e Merge all sources into a single source. 

In the same perspective as the sound recording, the video recording 
situations must meet several requirements for researchers to exploit. First, 
the record must be very good, so the image should not be underexposed 
or overexposed, it must include all the information related to the sound 
capture. For example, it can be detrimental for analysis to have the sound 
and not the picture or vice versa. Several cameras are sometimes necessary, 
when data need to be the teacher, class and several groups of students. 

Veillard (2013) analyze some research works in order to make the 
statement of video data collection methods. The characteristics of the 
situations studied (lessons in the classroom, lectures, practical work, 
interviews with teachers and/or students, or preparatory meetings between 
teachers) limit the variety of devices. He lists and describes just four 
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types of video-recording devices developed by researchers to film teaching 
situations. 

The first technical solution (for recording the entire class) supposes two 
cameras (a first camera equipped with a wide-angle lens on a tripod in a 
top-class area, with a wide static shot of the students and a second camera 
on a tripod in a corner in the back of the class with a static shot of the 
area around the table), a wireless lapel microphone worn by the teacher, 
one or more wireless lapel microphones worn by students to capture 
verbalizations in class (see the Figure 2). 


Legend: 
@ Teacher 


00 00 0000 00 © Student 
e Microphone 
| | == C Camera 


Fig. 2 - Video device for recording the whole class 


One version of this solution is presented in TIMSS project. The main 
idea in this project is to keep one static camera (on a tripod) and to use a 
second moving camera in two ways: 

e On the tripod but by allowing framing changes during recording (for 
example: track the movements of the teacher, or some students. 

e On the shoulder which allow to follow certain actors or certain artifacts 

e A second solution is a binomial device. The equipment necessary in 
this case is composed by a camera on a tripod, with a static shot of 
the pair or small group of students studying with, in the background, 
depth of field on the immediate environment of the group; one or 
more wireless microphones to the students in this group; a wireless 

microphone on the teacher (see the Figure 3). 

Another technical solution listed by Veillard (2013) is a mobile 
device for video recording adapted for kindergarten classes (as in French 
kindergarten classes) which are often divided into several areas where 
students are temporarily divided by type of activity. Because children are 
moving much from one to another zone, a camera up with wide static 
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Legend: 
@ Teacher 
o Student 
e Microphone 
C Camera 


Fig. 3 - Binomial video device 


shot is usually not precise enough to capture what is happening in a given 
area. In this case, a second camera mobile, shoulder, will allow following 
the movements of children, including outside the class if the educational 
activity considered the leads to it (see Figure 4). 


Fig. 4 - Mobil video device 


The last video recording device described by Veillard (2013) is 
a technical solution adopted for meetings or interviews (see Figure 5). 
The equipment is smaller and much easier to dispose in the room. It is 


Fig. 5 - Video device for meeting/interviews 
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important to have a camera on a tripod, with a still shot of the players 

present at the table; a microphone wired room on the table or lapel 

microphones for actors. An additional camera can be installed above the 
vertical to film the materials used, handling and registration. 
To synthetize, there are four modes to use the camera (see Table 1): 

e camera positioned on the tripod in static shot (no action on the camera 
which means no zoom, no movements) or in dynamic shoot (allowing 
zoom and movements in horizontal or vertical axis); 

e handheld camera (on the shoulder) without movements, zoom or with 
movements and zoom. 


Tab. 1 - Camera positions’ mode 


Static shot Dynamic shot 
Camera on tripod No action Zoom 
No movements Movements on horizontal 
and vertical axis 
Camera on shoulder No movements Movements 
No zoom Zoom 


Kilburn (2014) presents three methods for producing video recording 
within classroom settings depending on needs to capture or not more than 
one camera angle, to have or not the video available for immediate play- 
back, to have or not mobile equipment. 

A single camera recording is necessary if we not wish to capture 
more than one camera angle. In this case, Kilburn (2014) advise to place 
at the back of the classroom a HD digital camera with a wide lens angle 
for learners or an optical zoom for teacher. The teacher is the primary 
“subject” for the video recording. The camera may be positioned in front 
of the classroom to record students. In this situation, the teacher will be 
left out of the shot. 

Another alternative is to use a multi-camera recording for editing later 
(see also Veillard, 2013) or a live capture from multiple cameras. 

Advances in digital video technology will allow new methodological 
approaches or developments. The wireless connectivity allows video to 
be transferred or ‘streamed’ to a nearby device using a wireless (‘Wi-Fi’) 
network connection, reducing for example the need for obtrusive wires to 
be trailed around the classroom when undertaking a live capture recording. 

Smartphones and tablets are not only able to record video, but can 
also take advantage of the same sorts of wireless network connectivity 
discussed above to transfer video to other devices. In fact, software 
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developed for Apple devices even allows multi-camera recording from 
Smartphones or tablets connected to each other wirelessly. 

Wearable cameras, with ongoing improvements in the video quality, 
usability, and cost of ultra-compact wearable cameras, are bound to see 
more widespread use in the classroom. 


3.2. Advantages and disadvantages of camera’s types 


The camera on the shoulder is undoubtedly the one that allows the 
most opportunities: static shot throughout the registration or change of 
plan, zoom and camera movement, ability to remain static or change its 
position to better access certain events or follow the actors. However, 
it is technically difficult to implement because it requires knowing 
precisely what we try to decide at any time to his position, framing 
and plan changes, relevance to zoom in on an item. In addition, it 
requires significant expertise in handling the camera for good quality 
images (stabilization of the camera, taking account of the light sources, 
anticipating noise conditions, etc.). 

Hall (2007) is considering that “the job of a person operating a follow 
camera is to stay with the proxemics shape of the interacting group (1.e., 
bodies in relation to each other and things), ideally keeping everybody in 
that group within the visual frame as they move around. For example, a 
follow camera operator can attempt to have speaker and listeners in view 
as a speaker is making some point primarily with words. The reason for 
wanting to have the participants’ faces and bodies in view as much as is 
possible is that analysts will want to determine what people orient to in 
conversation (where gaze is allocated, how bodies are coordinated with 
media, etc.). But when the speaker begins to open a document to point 
out what he or she is talking about, or begins writing on a white board 
or sheet of paper, the follow camera operator can begin alternating 
between zooming in close to get the artifact-level details and zooming 
back out to get speakers and listeners. As a way to capture aspects of 
context that are (presumably) available to study participants, zooming 
in and out of the scene is preferable to panning across speakers and 
media” (Hall, 2007, pp. 9-10). 

The camera on tripod is probably more secure, especially when it 
remains fixed during the entire recording. Registering the class in a 
static shot wide and makes the open video for further analysis by other 
researchers. It is however not without drawbacks, especially if a player 
leaves the field, or if important information for the analysis remain 
inconspicuous (eg enrollments table). 
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Anyway, as we have seen in the types presented above devices, usually 
researchers combine several of them, using multiple cameras. This allows 
multiple viewing angles on the same object and the combination of a large 
and fixed plan and a more local and mobile plan. This is essential when 
the researcher wants to be able to capture many local scenes that take place 
in parallel: for example, a discussion of the teacher with a small group of 
students while other exchanges occur farther between, on the other students. 


4. Data organization 


Leblanc, Ria and Veyrunes (2013) propose the construction of an 
interactively “electronic corpus” to organize data, heterogeneous and often 
very large, using a spreadsheet and hyperlinks to the direct opening of 
various documents. The digitization of video recordings on the computer 
can then be used to fix the collected data (such cuts that appear on the 
tapes), to hid parts of the image (blur faces or silhouettes if it lacks 
permissions to shoot some people), to mix different records (if two or more 
cameras are used, a record can be embedded in another) (Veillard and 
Coppé 2009), to move scenes or images. 


41. Data Compression 


Because of their size, original video files cannot be kept in this form 
on computers (for reasons of space on device storage and facilities to 
handle them). Compression operations are needed to reduce this size. The 
multiplicity of formats (.avi file type, .mov, wmw, etc.), of video encoders 
(Sorenson codec type, mpegl, mpeg2, mpeg4), of audio (mpeg3, AAC, etc.) 
and of multiple adjustable parameters (flow rate, image size, etc.) do not 
facilitate this operation. A compromise must be found between the quality 
of picture and sound required for analysis, the smallest possible size of the 
video to allow manipulation and easier transfers. 

Here are many free or payable video converting applications (for 
example: Adapter, Compressor, Episode, Handbrake, Media Converter, 
MPEG Streamclip, etc). 


4.2. Methods of video-recordings analyze (Data Reduction) 


Veillard (2013) mention four methodological strategies used to perform 
the reduction of video data: 
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1. Observation strategy and systematic coding video recordings 
TIMSS Video Science (Roth et al., 2006) is an example of one type of 
methodology for the analysis of video recordings based primarily on 
coding categories. The objective of this project is to compare teaching 
practices in different countries (5 different countries: United States, 
Australia, Czech Republic, Japan and the Netherlands) and study their 
effects on learning. To ensure the solidity of this device in all national 
contexts where it was to be used, the researchers first looked for a 
strong consensus among the participants of these countries which was 
given by the use of the same codes. 
“Descriptions for each code were developed collaboratively as the 
group watched and discussed video examples together. Science Code 
Development Team members then independently applied the proposed 
definitions to a new lesson(s). Afterwards, the group compared their 
independent coding decisions and used differences in opinion as 
a strategy for clarifying the written definitions and for reviewing the 
effectiveness of the proposed codes in capturing the desired lesson 
feature. This process of independent review of lessons followed by group 
review and consensus building continued until 85 percent or higher inter- 
rater agreement was reached by the Science Code Development Team 
members or until a decision was made to drop, revise, or create new 
codes” (Roth et al., 2006, p. 7). 
The data reduction is performed by reference to conceptual categories 
developed by researchers. It consists, for coders, in searching 
the information flows, certain defined events or objects, which are 
indicators of the presence or manifestation of these categories, 
definition of the work of these indices and their application link with 
categories have already been created by the designers of the encoding 
device. The following analysis is quantitative and operates through 
statistical processing (descriptive statistics and cross-tabulations 
mainly). 

2. Crossing strategies for various types of descriptions 
Recently, many studies in comparative didactics (Schubauer-Leoni ef 
al., 2007; Sensevy, 2007; Sensevy & Mercier, 2007; Tiberghien et al., 
2007) postulate the interest of articulating several types of descriptions 
of the video recordings. 
On example is the thesis of Marlot (2008), based on the theory of 
joint action in didactics. The author favors a work by contrasted case 
studies. Two class sessions are filmed and analyzed. Video recordings 
are complemented by primary data associated: questionnaires and 
interviews with two teachers, pre- and post-test questionnaires for 
students. The data analysis process is operated in several successive 
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stages which mobilize different modes of description video recordings 

under different registers or genres of discourse (narrative, synoptic, 

categorical). 

. Progressive refinement strategy assumptions 

The methodology introduced by Engle, Conant and Greeno (2007) to 

study the role of discourse in conceptual learning is based on a method 

of data reduction operated by so-called progressive refinement strategy 
assumptions. This methodology is characterized by an intense use of 
video recordings at all stages of the analysis. 

It is successively used for: 

e select relevant passages for the object of study related to the research 
mentioned by a specific discussion topic; 

e characterize the phenomena by which manifests the object studied; 

e transcribe more finely selected passages; 

e code these passages with conceptual categories; search for factors 
explaining the phenomena highlighted and construct theoretical 
assumptions; 

e test and refine these assumptions on other types of discussions. 

This methodology needs an efficient indexation system: 

e “Searching for episodes of this topic was feasible because we had 
made content logs of the video-tapes in our collection” (Jordan et 
al., 1995). 

e “A content-log is written by someone watching a tape with only 
minimal reviewing in order to provide a time-indexed list of topics 
being discussed” (Engle et al., 2007). 

. À collaborative strategy researcher/actor observed 
As part of the action current, the theoretical point of view is to account 
for the asymmetrical relationship of an actor with his environment: 
he built his own world in the course of the action by selecting its 
environmental elements. The researcher is primarily interested in the 
pre-reflective consciousness, that is to say what makes a sign to the 
actor in the situation, his concerns, and that on which he focuses during 
the action. The data reduction work is strongly guided by the views on 
the action and is based on a methodological protocol where class video 
data is only an insufficient step to access this object. The researcher 
must rebuild the own world of the actor which it is not direct accessible 
for him. The researcher does not operate alone data reduction but in 
cooperation with one or more actors. 
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5. Procedures and tools dedicated to the analysis 


The TIMSS video project is dominated by an explanatory logic: the 
aim is to highlight the relations of statistical correlations between on 
the one hand, class configurations, shapes and teaching content and 
secondly, learning opportunities, with efforts to develop a common and 
uniform coding system for all countries and to ensure the highest possible 
reproducibility of video encoding process, regardless of the cultural 
context. 

Research conducted within the course of action is highly dominated by 
the understanding and focus on one or a few cases: it is about to focus on 
the perspective of the actor, on the meaning of the situation for him, on his 
own meanings. 

Engel and colleagues highlight a real dialectic between comprehensive 
phases (search for video segments with a subject-specific discussion) and 
explanatory phases (analytical and comparative approach, using criteria 
and encodings, distribution speaking turns, quantification of overlapping, 
types and number of outdoor activities in the discussion). 

Marlot uses the narrative register that refers to the idea of articulating 
interpretations and intentions of those events with more goals in the course 
of joint action. It also relies on a more explanatory language type analysis. 

Some researchers follow the idea theorized by Lemke (2000) that 
the didactic or educational processes are located in complex systems 
that require multilevel analysis of temporal extension. This is of course 
the case of research that mobilizes type analysis scales as macro, meso, 
micro (Marlot, Tiberghien and Malkoun). Other research, however, do not 
distinguish between different levels of analysis (this is the case of Engle, as 
well as the TIMSS project). 


6. Data reduction tool: macro analysis for indexing an event 
from video corpus 


It is a complete set of data that is aggregated for this project: 

e a questionnaire to assess the performances of in-service teachers on the 
concept of formative assessment; 

e responses of teacher’s questionnaire in each participating country; 

e sound recordings and videos of authentic teaching situations; 

e an indexing grid of sound and video recordings; 

e indexed sequences; 

e analysis analyzes of output files; 

e a method of selecting sequences of training for novice teachers; 
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e training course for new teachers; 
e pathways analysis. 

This paragraph presents the method for indexing an event from a video 
corpus. To ensure the reproducibility of the method, we conduct a four-part 


analysis: 


1. macro-analysis of all of the recorded footage to help identify formative 
assessment passages. 


Example: Video France, formative assessment phase 


Macro-Video-1 Analysis — Teacher, Daguerre College Shooting 14 


September 2015, 6th, Teaching hours: 8 am to 9:05 


Tab. 2 - Macro place of classroom activity 


Time Content Comment 
00:00 => 00 :02 Instructions: students take their They use the 
notebook exercises, recalling the video-projector 
exercises numbers, recalling the name to do the 
of the software to read a pdf, which correction 


student goes to the blackboard? 
Students raise their hands, one 
designated student, one student goes 
to the blackboard and corrects the 
exercise, teacher asks the class if they 
agree, the teacher asks who wants to go 
to the blackboard following the exercise. 


00 : 02 => 00: 04 


Teacher sends a student to the 
blackboard while this teacher walks in 
the class ranks, he looks at the books 
of students, one student corrects 

the exercice in the blackboard, the 
teacher asks the class if they agree, 
the teacher asks who wants to go to 
the blackboard to correct the following 
exercises, the teacher sends a student 
to the blackboard, teacher holds the 
graduated ruler because the segment 
or line tracing has its first high point 
on the board, the teacher asks the 
students to raise their hands, the 
teachers calls on one student, “student 
gives an answer, teacher asks «why», 
the sudent gives justification, the 
teacher asks the student questions” 
any difference between a straight line 
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Tab. 2 - continued 


Time 


Content Comment 


and a segment, students in the class 
raise their hands. 


00 : 04 => 00 : 06 


Teacher recalls “the last time we said 
that,” the students raise their hands, 
the teacher gives some definition, 

the teacher asks some questions at 
the blackboard, he gives the correct 
answer, the correction continues, the 
students raise hand, the teacher asks 
who wants to go to the blackboard 

to correct the following exercises, the 
teacher sends a student to blackboard, 
the teacher holds the garduated ruler 
for tracing the segment or line (because 
of its first high point on the board), 

the teacher asks the class if they 

agree with the student’s answer, the 
student gives an answer, teacher asks 
a question, students raise their hands, 
student gives an answer, teacher raises 
their hands, a student answers, and 
other students raised their hands. 


00 : 06 => 00 : 08 


The teacher asks another student, the 
student gives the right answer, the 
teacher valids, the teacher teaches the 
lesson. 

Next exercice, the teacher reads the 
instruction, one student raises his 
hand, teacher sends the student to do 
te correction of the exercice, from the 
student’s answer,the teacher gives the 
right answer , and he asks the class if 
they are all agreed. 


00 : 08 => 00: 10 


Teacher asks a student to do the 
correction of the next exercice, the 
teacher asks questions to the student 
at the blackboard. 

Teacher refers a student for the last 
two exercices, the teacher talks with 
the student about the rignt answer at 
the blackboard, the teacher asks the 
student to show him a segment, the 
student then corrects the exercice, a 
new request from the teacher about the 
straight line. 
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Tab. 2 - continued 


Time 


Content 


Comment 


00 : 10 => 00 : 12 


00 : 12 => 00: 14 


Teacher asks the class if they are all 
agreed. 

Oral Question, few raised hands, the 
teacher asks a student, a student 
gives an answer, teacher resumes the 
set, teacher gives the answer, teacher 
designates another student, student 
gives an answer, teacher gives the 
correction, last oral waiting matter, 
teacher asks a student, student gives 
an answer, the teacher valids the 
teacher asks a question, teacher takes 
over again and wrote on the board the 
right answer, another question for the 
student. 


The student gives an answer, teacher 
asks a question, student gives an 
answer, teacher asks another question, 
continuous interaction between teacher 
and student, validation request to the 
class, not back, question to the class, 
student gives an answer, the class 
validation request, 1 yes question to 
the class, student gives an answer, 
teacher designates another student, 
another student gives the correct 
answer, teacher resumes, he asks 

a new question, a student gives the 
correct answer, the teacher request 
validation of the class, teacher resumes 
and shows. 


00 : 14 => 00: 16 


Back to the lesson notebook. 


End of FAMTL 


To select a sequence, we recall the definition of the project (Black & 
William, 1998; Sadler, 1998; Gagatsis, 2000; Nicol & Macfarlane-Dick, 
2004; Black & Wiliam, 2006; Hattie, 2009; Looney, 2011; Cauley & 


McMillan, 2010): 


e the FA is an integral part of the teaching and learning process that 
contributes to its regulation; 
e allows teachers to identify weaknesses and strengths of each student, in 
order to reflect on their practices and adapt them to match better with 
the final goal of student learning; 
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e establishes a dialogue between the teacher and student, with positive 
fallen on educational. 
From a macro point of view, we observe a calibrated organization that 
evolves over time. 


Assumptions of this evolution: 

e exercises that present more and more troubles, no good answers from 
the students at the first time, need for reminder by teacher; 

e Time management (7 : 30 min => the issue of time management); 

e Organizational Management (ditto => 7 : 30 min => issue of time 
management); 

e observation of the number of hands raised => understanding indicator; 

e validation by the class. 
Confirmation/invalidation of these assumptions by micro analysis: 

e Time spent in the actions must evolve based on implicit or explicit 
criteria with respect to the teaching practice; 

e Repeat these characteristics at other times to become a characteristic of 
the teaching practice and gifts of teaching style (PISA). 


Oral interactions: 

e Omn to 07mn30: Teacher routine (who wants to correct the exercise at 
the blackboard, he sends a student to the blackboard (alternating girl/ 
boy kept during the sessions), the student brings a correction, validation 
by the teacher who asks the class if class is ok with him, then if the 
class does not agree, he can either ask the student to make a new 
correction or he asks the class) => co-Construction of enunciation of the 
effective knowledge correction; 

e from 7mn30s to the end of the video: teacher gives his own 
correction, because the majority of students sent to the blackboard 
have difficulties. He keeps the same structure, i.e. it is not the teacher 
who corrects directly, but it’s him who makes the final correction and 
institutionalizing knowledge. => Apparent co-construction, even share 
structure, but not the same relationship to knowledge; 

indicators: 

e number of raised hands; 

e quality of interactions with the student on the board; 

e quality of interaction with the class. 


Knowledge: 

e devolution => correction by the students on the blackboard with 
behaviors which change over correction; 

e institutionalization => always made by the teacher with the class 
validation; 
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didactic contract: very established: hands raised, correction on the 
blackboard, alternately girl/boy, classroom organization. 

Teacher observation: 

in the first 2 minutes of the correction step, the teacher goes into the 
ranks and looks at the student booklets; 

second passage, with indexing TIME OF ASSESSMENT, TOOLS/ 
STRATEGIES PHASES and to make a micro-analysis and confirm or 
refute the hypotheses constructed from macro-analysis; 

third passage for indexing content with mathematics. The third passage 
is to frame the mathematical content with actions and the application 
context. This passage can check also the first encoding; 

fourth crossing. This passage is made by someone other than the one 
who conducted the other three passages to compare the two micro- 
analysis. 


7. Using video in a training program: example of France 


Concerning the French case, it was decided from the outset that the 


training program would be short, available at a large number and at a 
distance. It is in this perspective that an MOOC type device has been set 
up. The two main reasons for the implementation of this type of device 
were: 


the training models and therefore the program is national. It is therefore 
not possible to modify a training module to introduce a distinctive 
training theme, at the risk of not offering the same training between two 
teachers in training; 

as part of a national program to increase the digital competence of the 
population, several devices are put online in a volume of 3h, 6h or 9h. It 
is a question of being able to deal with a particular point of the program 
or a pedagogical practice in the given time and in a type of training of 
a professional type. 

Presentation of the platform and its contents: 

The platform used to operate the training videos in MOOC mode is 
OpenEdx®. It offers a structure allowing easy reading of contents. A 
vertical menu offers all the training modules and a horizontal menu 
offers the content backed up by each module. Two modules have been 
created, namely: 

formative assessment; 

links to research. 

The content of the module focusing on formative assessment is 
composed of: 
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a positioning test allowing each learner to measure its proximity 
between its representation and the definitions proposed by the 
consortium; 

knowledge text proposed by the consortium; 

videos to understand the topic; 

videos to show authentic class situations; 

a knowledge base that provides a synthesis in 4 pages. Its role is to 
set the necessary scientific resources for the learners; 

a knowledge test, the questions of which are drawn from the 
questionnaire drawn up by the consortium. 


The module whose title is links to research, aims to offer learners 
complementary links from the research, the link of the website of the 
project and a survey of satisfaction of the device put online. 
2. Analysis of the device 

Given the main theme of this chapter, namely video, we will give here 
some details on the videos present in the online platform. Regarding the 
section, videos to understand, there were 4 videos available free on the 
Internet which presents a point of view on the formative assessment: 


the first is proposed by Marc-André Lalande, consultant, Canadian 

expert teacher; 

the second proposes an adapted iconography of some rules of 

formative evaluation; 

the third describes the types of feedback the teacher can do to his or 

her students; 

the fourth objective is to present the speech of a researcher, former 

rector, Jean-Marc Monteil, who talks about the victims of the 

assessment. 

The second section proposes to show authentic situations: 

— the first video features an explanatory animation; 

— the second video proposes a no-grades approach, without 
quantitative assessment; 

— the third video presents a participatory approach. 


The third video is a video from our data. This video was selected 
because it reflects a point of difficulty for learner teachers. We describe 
here the process of selecting this video data. 

Firstly, the consortium has established a questionnaire to reflect the 
difficulties and representations of novice teachers in training «beginners». 
After analysis, several difficulties and conceptions were identified by the 
researchers. It was then decided to record using a background camera 
with a wide angle and a microphone on or in the direction of the teacher. 
The goal is to have a comprehensive view of the classroom and to hear 
very clearly what the teacher could say. A macro analysis was carried out, 
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making it possible to select extracts with particularities either with respect 
to the formative assessment or with regard to the difficulties identified by 
the initial questionnaire. 

In our case, a major difficulty is the management of time and interactions 
in a phase of putting students to work and evaluation in a formative 
assessment process. 


VOIR L'UNITÉ DANS STUDIO 


INFO DE DEBOGAGE POUR L'ÉQUIPE PEDAGOGIQUE 


Proposition d'une analyse macro de la vidéo, son syllabus : 


Temps/Durée Contenu des actions Analyse | 


00 :00=>00 :02 | Consigne : sortir son cahier des exercices, rappel des 
numéros d'exercices à faire, logiciel pour lire un pdf, 


qui va au tableau ? 


L'organisation et la consigne => issue de la 
préparation de l'enseignant, pratique hors clase 


Fig. 5 - A video to understand the management of the regulation of interaction 
during a formative assessment process 


About 125 participants (teachers at the middle school and at the high 
school) to the pilot training course were asked to fill out electronically the 
initial questionnaire (test de positionnement-premier test du dispositive 
de formation) about their beliefs on formative assessment. We use the 
software of survey, Sphinx® and the Platform, OpenEdx®. The duration of 
the training is 6 hours online. The teachers could debate before and after 
the pilot course through the forum (in the platform). And there is also a 
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chat online (live) so the teachers could express themselves and debate and 

talk with each other and also with the trainers (the designer of the pilot 

course). 

The meaning of the formative evaluation for the teachers is well known 
and recognized. 4 categories of answers are obvious: 

e Category 1: assess student achievement, identify objectives, allow 
students to have a description of their difficulties, give teachers a 
chance to check the validity of their work, give teachers a chance to 
understand the changes to be made for teaching methods. This category 
never totally disagrees with the assertions. 

e Category 2: assess students’ abilities to use mathematics in everyday 
situations, authorize and apply comparison of student outcomes 
(strongly disagree); Permits to write a classification of the students 
according to their performances. This category covers the whole 
spectrum of opinions with a strong disagreement on the assertions: 
allows and compares students’ results. 

e Category 3: Based on the work process of students more than on the 
results in mathematics, provides feedback that enhances motivation and 
leads to improved knowledge and abilities very strong majority strongly 
agree. 

e Category 4: Allows you to write a classification of students according to 
their performance. Very little disagreement. 

These 4 categories provide information on the representations of MOOC 
participants who responded to the questionnaires. First of all, they have at 
least basic knowledge on what characterizes formative assessment, namely 
Cat 3: initiate a process of assessment of the process of constructing 
meaning and provide feedback to students enabling them to remain 
motivated and thus encourage the scientific approach by a set of games- 
error-successes. It is more difficult for these teachers who responded to 
have clues as to what this formative assessment gives them, namely Cat 
1: level of student acquisition, identification of objectives... The sample 
is here more divided, the majority that are agreed with the assertions, but 
there is also a set that rather disagrees with these principles. Symmetric of 
Cat 3 on an assertion, there is strong disagreement with the assertion that 
permits and applies a comparison between student outcomes. 

In conclusion, in-service teachers who participated in the MOOC and 
who responded are sensitive to the question, are aware of the interest 
of the approach but do not necessarily have all the indicators that this 
assessment approach can bring them in the management of students’ 
meaning construction. 

However, the teachers do not feel competent enough to submit activity 
where students realize a peer assessment but they do feel competent when 
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they talk to students one by one and explain them the instructions to 
follow. By contrast, the teachers do not feel competent enough to identify 
and make the students identify their own abilities and weakness. They are 
more facilities to teach to the whole class than a peer group (or a small 
group of students). 

By combining these first two analyzes, we come to a first outline of 
the categories of knowledge and skills of the teachers (responses). They 
have knowledge on the subject, are able to identify what this can bring 
them as well as to the students, but with an uncertainty as to their ability 
to understand and use the indices that are provided by this method. This 
second point is precise here where it is a question of implementing an 
activity in which the metacognitive skills of assessment of the other or 
even autoscopy for purposes of personal construction and meaning, here 
in mathematics. They seem comfortable to help overcome an obstacle to 
students by bringing the solution themselves, or exercises. This seems 
more difficult for them when it comes to implementing an approach that 
allows the introspection of the difficulty so that the student develops 
strategies on his own to 1) analyze the reasons for success or failure 
strategies and 2) enabling them to re-engage in other situations and thus 
consolidate the notion of scientific approach. 

They also follow a pattern: prepare, implement, observe and analyze. 
The teachers also replied that the formative evaluation allows the students 
to know the points on which they have to rework and the teacher to 
know in which area/competence to help the student. Before doing the 
questionnaire, the teachers wrote that they teach by using the students’ 
mistakes. The teachers preferred the group work (students work together). 
The teachers also used oral assessment through a quiz and the students 
have to answer according to the answer given by the other. 

Before the administration of FA: the teachers checked that the students 
understood well the goals of the FA (ask questions to the students). The 
teachers chose to teach some lesson and examples that will be present in 
the FA. The teachers were very careful to the behavior of the students and 
the process used by the students. They always corrected the work done by 
the students, and administer some feedback. 

In general, teachers use all the activities mentioned to implement a 
formative assessment. The only case where it is ever recorded is when 
students set up a mathematical problem starting from the text. 

In summary, we can make some assumptions about the representations 
of the French teachers who made the MOOC and who replied: 

e theoretical knowledge of the concept of formative evaluation: from the 
point of view of the definition and the contexts with which it can be 
implemented; 
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e difficulties for teachers to learn from the evidence provided by the 
process; 

e difficulties for teachers to implement meta-cognitive skills that allow 
students to assess their difficulties and successes and implement 
strategies to accomplish the task; 

e the causes of students’ difficulties are well identified and some are 
discarded: such as the limited abilities of the students or the knowledge 
taught previously. 

Therefore, we can retain that the teachers who did the MOOC and 
answered the questionnaire has competences on the topic but have difficulties 
with the indices of construction of meaning that can bring the method. 

In the training process, the tool used, here a MOOC, is above all a 
revealer of the representations and conceptions of teachers in training and 
in service. It is like the questionnaire, namely a declarative tool. We used 
two types of videos: 

e videos to understand; 

e videos to show. 

The structure of the MOOC and the particularity of the videos had only 
one aim: 

e the positioning test: to question the learner, explain to him/her at the 
beginning of the training the specific topic on which he/she was going 
to work, namely here the management of time and interactions in a 
training assessment process; 

e videos: to make concrete, to have a real representation of what was 
evoked in the positioning test. The two types of videos aimed at 
making the link between theory and reality of an authentic situation. 
Here, we place ourselves in a modeling activity: a modeling activity 
with a distinction between theory and events and learning is mainly in 
the relationship between the two (Tiberghien, Vygotski); 

e a knowledge base as a scientific text, a theoretical reference which 
is intended to be sufficient for itself to enable the learner to evolve 
these representations. Here again, the principle of modeling activity 
resides with a process of the type: appropriation of the theoretical 
model, understanding of the reality of the event by manipulation of the 
theoretical model and learning when it is easy for the learner to do of 
going back and forth between these two worlds: theory/reality; 

e a final test to allow the learner to measure the evolution of his or her 
conceptions and also the gap that may still exist between conception 
and theory; 

e in the perspective described above, it is therefore necessary to have a 
strong epistemology regarding video data: what will be recorded, will 
serve in what purpose (why) and when? 
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8. Conclusions 


We have worked on this chapter following three approaches. The first 
aimed at taking stock of the notion of video data, the second proposed 
a state of the art in the use of this type of data in research in the arts, 
humanities and humanities fields, exemplify the method and use of data in 
the project. 

From the first two approaches, it follows that there is no unification of 
practices with this type of data. Starting from various works, including 
TIMSS-Video (1999), the consortium worked on the harmonization of 
a practice, namely that of data acquisition, micro analysis, and hence 
indexing in the Web directory and the fine analysis allowing each one to 
make the link between conceptions of the teachers and objectives of the 
training set up. 

Despite the Dublin Core standard descriptors, there is still some work 
to be done in harmonizing to allow countries to use videos other than 
the one they have produced themselves. Due to time constraints, it was 
not possible to carry out cross-analyses of the type: 1 video produced 
by country X is analysed by country Y, then coding of the two analyses 
is carried out, and a comparison, with the consequence that if and only 
if a small difference is found, then the corpus and its analysis can be 
exploited by the whole community. At this stage of the project, each 
country exploited, according to their identified needs, the videos they have 
produced, analysed and selected. Several prospects for continuing work 
can be envisaged, namely: 

e Have the same objective of training, videos from different countries 
and to take cultural differences or not? What impact will they have on 
education? And on formative assessment? 

e From the point of view of the training system, the impact of face-to- 
face and distance training? Regardless of the country? 

e From the point of view of data collecting, each teacher can use a 
smartphone to record, publish and work with others, how is this 
modality taken into account in training? 

e From autoscopy to experience sharing, using video tools and shared 
with digital tools, how to create a catalog of professional gestures, 
guaranteeing a diversity of practices and also of an era? 

The last point on which we want to react here is the very notion of 
video in training, already extensively dealt with, but in a different format, 
renewed by the MOOCs. It turns out that 90% of the participants in the 
MOOC appreciated the format with claimed autonomy of learning, a kind 
of “tutorial of existing practices”, which answers a very precise question 
for France: how teachers regulate interactions and time in a formative 
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assessment phase. It is not a question of working on the whole idea, but 
of looking at a specific point of the problem. The video as a surgical 
descriptor of a practice? 
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6. From beliefs to practices: 
the video-analysis methodologies 
to observe the formative assessment 
in classroom 


by Stefania Lovece*, Ira Vannini* 


1. In classroom practices and teachers’ reflective thinking 


OCSE-Pisa international data from the last 15 years — but recently also 
the Talis survey (OECD Talis, 2014; Vieluf, 2012) — have shown how the 
use of more or less innovative teaching and coherent formative assessing 
practices is mostly determined by the teacher variable, more than by the 
geographic or organizational variables of the school. 

This is especially true for Italy, where both the analysis of the students’ 
learning outcomes and the statements made by the teachers themselves, 
show considerable differences within a same schools (between different 
classes and, then, different teachers). 

This tendency to individualism and self-centeredness of the teachers 
is often accompanied by strong deficiencies in collective and democratic 
work among teachers. 

In this perspective acting on system variables that aim at changing 
the culture of school evaluation is still extremely important, both in Italy 
and abroad. At the same time it is as much essential that we focus on the 
teachers’ practices in the classroom. 

Ever since the 80s and 90s, studies on teacher change have highlighted 
how changing educational practices is a key factor in changing the 
beliefs teachers have. On the other hand, teacher training paths that aim 
at changing practices by only working on beliefs often appears to be 
counterproductive (Guskey, 2002). 

Therefore, in order to encourage the teachers’ reflective thinking, their 
capacity for self-assessment and critical analysis, it is necessary to start 
from their individual practices. 


* Alma Mater Studiorum Università di Bologna. 


96 


Copyright © 2018 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788891774637 


It is a question of encouraging teachers to commit to change, to exercise 
their reflective thinking (Dewey, 1910) and to evaluate (and if necessary 
modify) their actions as they would evaluate a scientific hypothesis. 


2. Observing practices as a fundamental tool 


From this theoretical-methodological perspective, the observation is 
a fundamental tool to help in-training teachers to ‘start from practice’ 
(Danielson, 2007). Observation draws attention to empirical data made of 
‘actions and behaviour’ in an actual framework. In this way teachers are 
able to compare the beliefs they have with empirical data and use them to 
formulate and reformulate new beliefs. 

Data that emerges from systematic observation has the potential to allow 
the observed subjects to distance themselves from their actions and to be 
able to think critically about them. Data, if it is collected diachronically, 
also allows subjects to take note of the progress made in time and to make a 
diagnosis, serving as a starting-point for projects for change. 

Obviously, knowing the observations’ objects and criteria is a fundamental 
condition to get teachers to be actively involved in a virtuous training cycle. 
We speak for a training cycle in which the teachers themselves can analyse 
their own practices and the practices of others teachers, changing their 
point of view and modifying their ‘routinely practices’. 

In terms of changing beliefs, observation becomes even more significant 
when it puts teachers in a participative position (Danielson, 2012). This 
happens, for instance, in peer observation: teachers take turns in being the 
observer and the observed (Bell, Mladenovic, 2008), watching themselves 
from the outside and watching from the outside the practices of others. 
The goal is to activate their reflective thinking, their ability for collective 
discussion and for making democratic decisions (House, Howe, 2003) that 
aim at planning better ways to act in a teaching-learning environment. 


3. Video analysis to enhance the classroom observation 


The use of video-analysis for teacher training (Ferretti & Vannini, 
2017) goes hand in hand with in classroom systematic observation and with 
the observation of teacher behaviour especially. 

Many studies still show the effectiveness of the microteaching 
(Calvani et al., 2011) and, more generally, of the teacher observation with 
consequent video-analysis to promote change and improving of teachers’ 
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professionalism (Rossi et al., 2015), namely with math teachers (Casabianca 
et al., 2013; Walkowiak et al., 2014). 

These strategies create a strong connection between theory and 
practice and they allow in-training teachers to change their point of view 
and to observe themselves in video sequences (Altet, Charlier, Paquay, 
Perrenoud, 2006). 

These methodologies that enhance our observational skills through 
the use of video are currently largely used abroad: the American and 
the British research (Guernsey, Ochshorn, 2011) highlight the need to 
use observation to make teachers more and more aware of their actions 
when teaching. Other experiences have been made by some American 
professional associations (such as the New Teacher Project, the New 
America Foundation and Teach Stone) which are developing systematic 
tools to observe the behaviour of teachers in classroom. 

More specifically, Santagata’s studies (Santagata, 2010; Santagata et 
al. 2010; 2011) regarding the use of video-analysis are a very important 
reference for teacher training. For teachers, the careful observation of 
students during their learning process and of how teachers behave in 
relation to that, are key elements to exercise their analytical skills and 
reflective thinking. 

Moving in the direction of creating opportunities for shared analysis 
and discussion about specific events that happen in the classroom are the 
American ‘Video Clubs’ (van Es & Sherin, 2010) and ‘Lesson Studies’ 
(Lewis &Takahashi, 2013) which are the result of Japanese studies and 
work towards using observational data collected during classes to discuss 
and collectively re-plan — between observed and observer — the teaching- 
learning processes. 

The research carried out in French speaking countries is also of great 
interest. This research mainly focuses on the teachers’ self-assessment 
processes (cf. Laveault et al., 2009; Paquay et al., 2010), on studies about 
teaching practices (Altet, 2003; 2006; 2012) and, more specifically, on 
“action analysis” (Durand, Filliettaz, 2009). 

It is qualitative research that analyses “activated” teaching (Iobbi, 
Magnoler, 2015) in order to identify its key elements and draw from them 
opportunities for teachers to use their reflective thinking and enhance the 
self-evaluation and critical skills of teachers. In all of this, videos are a 
tool of great value to foster the teacher in his thinking about the action 
through different methodological approaches (Theureau, 2006; Vinatier, 
2009). 

The connecting suggestion of these different strands of research — 
research stemmed from different cultural backgrounds and different 
theoretical and methodological framework — is to encourage the 
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development of teacher professionalism helping teachers themselves to use 
their analytical thinking on their practices and the practices of others. 
Focusing on the details, paying attention to what one is doing inside 
the classroom (a place that is traditionally not open to outside observers, 
whether they are human observers or video cameras) is what allows 
teachers in training to really notice what they are doing, rethink about it 
and distance themselves from it so that they can then be able to criticise it. 

In summary, the research on video-analysis (in classroom) highlights 
how this methodology — if it is integral part of intentional training 
pathways: 

e promotes the reflective thinking of teachers and better responds to their 
training needs (Meyer, 2012; Ertmer, Conklin, Lewandowski, 2002; 
Plakhotnik, 2001; Mottet, 1997); 

e allows teachers to identify strategies for self-improvement (more than 
what the analysis of the students’ learning outcomes itself does) (Meyer, 
2012; Ertmer, Conklin, Lewandowski, 2002; Plakhotnik, 2001; Mottet, 
1997). 


4. Video analysis and Formative Assessment 


Concerning classroom assessment practices, international research 
(Harbor et al., 2015) highlights the importance of analysing teacher 
attitudes in the use of formative assessment. 

Video analysis urges teachers to “see” what is happening in the 
classroom, and in particular to focus on teacher assessment behaviours 
that have a positive impact on learning outcomes. Specifically, three 
types of core teacher behaviour related to formative assessment and 
teaching individualization are highlighted: how it involves students, using 
questions that create opportunities for students to intervene and respond 
(Kern & Clemens, 2007); how to use the modelling behaviour; how it 
communicates formative feedback. 

In this sense, video analysis follows a very similar method to that the 
one used in the research experience FAMT&L: the aim is to draw the in 
training teachers’ attention to the details of those behaviours that are more 
related to educational effectiveness. 

Being able to describe these behaviours, to see the details and to re- 
think them in an improved way, are very valuable training opportunities 
for teachers. Research carried out by Kane ef al. (2011) has attempted 
to explore the connections between teaching practices and student 
performance. It is precisely from these studies that emerges the need for 
specific observational procedures to focus on the actions that teachers take 
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in the classroom, in particular on assessment practices. This is important 
both in the early phase of collecting information on students” learning, and 
in the specific phase in which the teacher is engaged in giving formative 
feedback to the students. 

Observation in the classroom aims to detect the assessment strategies 
used by teachers during the teaching-learning process: 

e how to communicate assessment objectives and criteria to the students; 

e how to present and deliver assessment tests; 

e how to collect information about the students’ achievements in the 
classroom; 

e how to correct assessment tests and errors; 

e how to communicate feedback to the students. 

The experience in the schools shows that FA is not a natural habit of 
the teachers and there is a need to improve the teacher practices on FA. 
To do this, it is essential to have in-depth knowledge of what happens in 
the classes during the assessment moments in order to identify “good” 
and “bad” practices and thus design effective teacher training and staff 
development courses. Therefore, the FAMT&L Project has acted in this 
perspective: the analysis of the specifics of teacher assessment behaviours 
and the use of video recording in the classroom have been the key tool for 
designing professional development pathways and to produce a real change 
teacher. 


5. Pilot course for FAMT&L project: a common training 
model for all Countries 


In line with the theoretical and methodological studies briefly described 
so far, in the FAMT&L project one of the main goals achieved was the 
design of a model course for mathematics teachers in which the use of 
video and observations in the classroom could be the main method to 
promote reflective activity that could generate change in teaching practices. 

A model course that could adapt to the different designs of the partner 
countries — differences dictated by different timings, different trainees’ 
selection modes, different certifications, etc. — did not prevent the specific 
courses to be characterized by some common features. 

It has thereby been possible both to preserve each country’s 
specificities, and to have a similar architecture for monitoring the training 
programs using common survey tools in order to allow evaluation and 
comparison. The main characteristic of these courses was the use of video 
in the training sessions for teachers in service, following guidelines from 
the scientific and theoretical framework. In particular, the use of video has 
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been incorporated as an opportunity for reflection and discussion on the 

construction of beliefs and on formative assessment practices. 

A further important aspect is the traceability of the change in these 
beliefs and practices in the perspective of improving teaching in an 
innovative way. 

Regarding the content of the course, the training program provides 
general knowledge on teaching design and assessment and more specific 
knowledge of mathematics teaching with particular attention to summative 
and formative assessment and therefore to “for” learning assessment. This 
is because we thought that a correct use of FA methods and techniques is a 
key element to make teaching math more effective and innovative. 

To develop a training course that could enhance the reflective activity 
of teachers as a potential key to change in future teaching practices, 
some learning activities have been suggested trying to combine 
theoretical content and teaching methods based on experiences geared 
towards elaborative and reflective action supported by the competent 
intervention of university teachers, supervisors, tutors or critical friends 
(experienced teachers and already partially trained on the subject who 
participated in the course). The suggested activities of communication, 
analysis, discussion and reflection centred on issues related to the design 
of assessment practices, the development of data collection tools, the 
provision of concrete experiences and practices, the documentation of 
the path, have been tested through a number of stimuli and reflections/ 
discussions shared in presence and at a distance with the tools and 
materials on an e-learning platform. 

Some of these activities are described below. 

e In-depth study of the themes presented in class, in which the main 
theoretical and methodological references on the assessment and 
teaching of mathematics have been discussed, investigating in particular 
the theme of methodologies and tools to be used. 

e Observation activities during which videos from the web repository 
that have previously been analysed are shown or short videos are 
systematically analysed using the shared observation grid (or selected 
parts of the grid) previously used for analysing the collected videos. 

e Reflection on one’s own evaluation practices in the direction of 
planning a new evaluation strategy in which, after dealing with the 
issues and the various evaluation models, teachers participating in the 
course are required to plan the activity they will eventually lead in 
future. 

e Sharing of the analysis and the activities planned in order to 
collectively reflect and gain greater awareness through the feedback and 
reflection of others. 
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By themselves, all these activities, however, are not sufficient to 
give life to the experience of change in the evaluation practices of the 
teachers concerned and therefore it is particularly important that they 
are accompanied and supported by instruments and trainer serving as 
guidance and orientation. Recent research has shown the critical role 
of tutoring or mentoring to stimulate learning from experience and in 
the experience highlighting how professional performance improves when 
teachers are listened by a mentor that gives them the ability to create 
connection points with their own experience (Filliettaz, 2014). 

In this sense it is also important to emphasize the use of strategies 
and methods drawn from research experiences that show how the 
video recording of educational practices becomes even more incisive 
if intertwined with opportunities for reflection and dialogue urged by 
questions-stimulus introduced by a researcher/facilitator or by tools such 
as drills and tasks with questions stimulation or learning diaries that 
guide the subject to a metacognitive level (Santagata, Zannoni, Stigler, 
2007). 


6. The use of video for teacher training between theory 
and practice 


The methodological and operational implications for teacher training 
have allowed us to better identify ways to use videos on assessment 
practices in the training paths we have proposed and in particular to 
use videos as stimuli to conceptualize and reflect on the teachers’ own 
practices. Additionally, we have found especially important to use video 
sequencing to allow us to identify more accurately the actions put into 
practice by teachers during their classroom lessons. 

The videos used were obtained the phase of the project in which we 
investigated teachers’ beliefs and assessment practices (see chapter 4). 

The correlation between the research on conceptions of teachers’ 
beliefs and the first phase of observational research has allowed us 
to understand what types of wrong beliefs teachers had on formative 
assessment in the classroom. What has emerged is a common tendency 
to use traditional practices of summative assessment and difficulties in 
perceiving formative assessment as a useful tool for improving teaching 
and learning. 

Difficulties to understand the effectiveness of formative assessment 
were also found during a first analysis of the first videos examined. Many 
of the natural situations of math teaching in the classroom confirmed the 
results of the first phase of research (the questionnaires), as they have 
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highlighted the use of evaluation mainly aimed at “measuring” learning 

and assessing them in a summative way by using not-rigorous methods of 

collecting information about learning and not accompanied by a correct 
use of feedback. 

Clearly, to get more detailed information for our research, we had to 
build a tool for the observation and analysis that we could use both to 
analyse recorded videos in the classroom and to categorize and store video 
sequences to be used in future training courses. The tool has been defined 
with reference to guidelines derived from international literature and 
experiences of in-service training. This tool has been useful in gathering 
several indicators on good and bad mathematics teacher training practices 
(e.g. their habits in gathering information on student learning; the error 
correction process and the use of feedback to support learning) (Lovece, 
Michael-Chrysanthou ef al., 2016). 

First of all, we started to consider what the literature on docimology 
defines as the classic phases of a proper evaluation process. This is merely 
to refer to specific technical and procedural guidelines which, if they 
refer more specifically to the summary (objective and most scientific) 
assessment procedures, can also be considered valid for the practice of 
formative assessment. The stages of this process we refer in particular are 
(Gattullo, 1967): 

1. the definition of the object, as the pupil performance manifest following 
the teaching action, described and identified in terms of indicators 
“capable of providing detectable data on learning and its outcomes” 
(Tornar, 2001, p. 147); 

2. measurement, that is the assessment, detection of data capable of 
providing information on the level of learning achieved by a pupil. 
This phase must necessarily have the validity requirements (when the 
object identifies with what is actually measured) and reliability (when 
the information it provides is responsive to the actual status of the 
measured object and therefore the measurement is repeatable remotely 
time or other subjects with constant results); 

3. the assessment, that is the expression of a judgment on data and 
information obtained through a particular language and on the basis of 
a specific. 

They are also taken into account the theoretical and methodological 
guidelines for measurement practices in a way more properly formative as, 
for example, the systematic use of the on-going evaluation of the progress 
of the students, the communication of the training feedback, the error 
analysis, etc. 
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6.1. The grid: a tool for video analysis 


The tool created starting from these theoretical-methodological 
references has therefore been used to provide each video with metadata 
that allowed categorization and sequential descriptive analysis (to observe 
correlations) that helped to define the classroom assessment practice 
profiles. 

Therefore, the grid used has been structured on different levels. On 
the first level we find the data that “identify” the video files which allow 
for first storage: the video identification code; Nation; Language; Type: 
audio/video (length, format); Creation date; Author; Class and school level; 
Number of students in class. 

On the next level, we have used categories which have allowed us to 
better study the qualitative analysis as we considered the many variables in 
play in such a specific and complex process as assessment. 

From an environmental perspective (Bronfenbrenner, 1979) this 
observation grid allowed us to collect several indicators on the evaluation 
practices of math teachers and to group them into five macro-categories: 

1. math content (content and skills that are the subject of teaching); 

2. assessment time (before, during or after a specific learning activity); 

3. assessment setting (with all the students in classroom, with student 
groups or with each single student); 

4. type of tools used to collect data on student skills (written tests, oral 
exams, behavioural observation, ...); 

5. stage of formative assessment (presentation of the assessment activity, 
information gathering, error correction, feedback). 

In the first category we consider the information about the 
mathematical content that is the subject of each lesson/situation. Taking 
into account the complexity of the teaching-learning process, the activities 
in this category obviously cannot be considered only as content in 
mathematical knowledge (mathematical objects) but we had to expand our 
horizon to take into account the skills and competences the students bring 
to the learning process. So we adopted a two-dimensional content/capacity 
schema based on the OECD-Pisa approach (OECD-Pisa, 2013) as shown in 
Table 3.1. 

Other categories have been elaborated starting from the consideration 
that, as has been said, a good assessment strategy must necessarily 
refer to an equally rigorous measurement phase. Only in this way, the 
assessment can be defined strictly scientifically, and flexibly adaptable to 
the operational choices that the teacher adopts for a successful process of 
teaching/learning. 
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Tab. 1 - OECD-Pisa Approach/Capacity Schema (OECD-Pisa, 2013) 


Contents Skills 
Numbers Communication 
Spaces and shape Mathematization 
Uncertainty and data Representation 
Relationships and Functions Reasoning and argumentation 
To devise strategies for solving 
problems 


Use symbolic, formal and technical 
language and operations 
Use math tools 


In particular, thanks to the collaboration with the teachers, we 
considered the importance of implementing choices that would ensure the 
widest and most adequate framework of reliable and accurate information 
on achieving specific learning objectives. 

In the grid, the second category (time) was set to identify the moment 
the formative evaluation activities are carried out over the time of the 
whole lesson (for example before, during or after a teacher’s explanation). 

The setting category (third) considers the context of training evaluation. 
This is necessary since categories, time and space/context (which also 
take into account the predisposition of the class group) are variables that 
can affect the teaching process and, if pedagogically planned and suitably 
adapted to the specific teaching-learning situation, they can play a very 
positive role in facilitating the process. 

The next category, the one that considers tools, is equally important. It 
is also through the use of appropriate tools and techniques for detecting, 
processing and analysing the information that can be obtained, which 
can best integrate evaluation practices into strategies and didactic paths 
promoted (Domenici, 2003, pp. 34-37). This involves a strong focus on the 
validity and reliability of the information to be taken into consideration, 
as the teacher should be able to express a reasoned and documented 
assessment of the pupils’ performance (Tornar, 2001, p. 161). 

Docimology studies have produced several classifications of test 
and assessment techniques that can be used in the different contexts 
and moments of a training path. There are also numerous studies 
demonstrating the validity and limitations of the most commonly used tests 
in teaching practice (Vertecchi, 2003). 

The variety of tests available is very wide and often the choice of 
the tool and its construction is very difficult for the teacher. This also 
because it is very much dependent on the purpose for which an instrument 
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is used and from the moment when an assessment is made. In the grid, 
instruments have been classified into types of tests, distinguishing between 
scripts and oral, individual or group, and structured or not. All those 
situations in which a teacher implements an assessment process, even 
though this practice does not follow the formal process, is based on 
spontaneous or unexpected observations with non-systematic or informal 
interactions. 

Finally, the last category of the grid is the most interesting and most 
characteristic because it collects a wide range of behaviours and actions 
that can be considered as indicators to be observed in the various phases 
of the assessment practice. It is also the most complex category, because it 
collects actions and behaviours (predominantly of teachers) implemented at 
different moments of the assessment phase. 

For this category, reference was also made to the methodological 
indications from docimology studies, foreseeing phases of which should 
be a good assessment practice. For a systematic and rigorous assessment 
methodology, it is necessary to follow a series of indications to try to 
reduce the risk of subjectivity. At the same time, it is necessary to ensure, 
as much as possible “formative” assessment that ensure a useful feedback 
to adapt teaching and learning strategies with their aim. A proper “way” 
to activate a correct evaluation is the one that distinguishes the phases of 
(Domenici, 2003; Vannini, 2009): 

e presentation of the test, in particular by sharing with the class of 
correction and/or assessment criteria; 

e the administration of the test, which requires special arrangements 
depending on the type of test used, for example written or oral; 

e feedback, that is, returning the results obtained by the student. 

For each of these phases, therefore, in grid construction, we tried 
to identify all observable actions that could be considered as indicators 
of good or bad assessment practices. For example, at the stage of the 
presentation of the test, the teacher should clearly communicate when the 
pupils will go and share the goals, the contents, the methods of testing, the 
criteria for correcting it and assessing it. 

In the administration phase, however, actions may differ according to 
the type of test used. For example, particular attention should be paid to 
the distribution of a written test, providing explanations on how to run the 
test or compilation of the distributed module and strategies used to ensure 
that there is no interference in individual work (for example, to avoid 
some copied from class mates). In the case of oral tests, the operating 
instructions are different, depending on whether they are individual, group 
or paired, and depending on the mode and timing for example to answer a 
question. 
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In the recording phase, the detectable and observable activities are 
few and differ according to the type of test performed. For example, if 
the teacher decides to assess a student through direct observation, he or 
she can decide whether to use more or less structured grids or a freer and 
narrative recording mode. 

The feedback phase is also quite articulated and includes all the actions 
through which the teacher can give information on the result achieved 
and aimed at allowing each pupil to be aware, for example, of the type 
of error committed. Even at this phase, actions may also be differentiated 
depending on whether it is a written test or an oral test or even a test 
involving peer evaluation. 

In this chapter, unfortunately it is not possible to describe in detail all 
the actions that can be detected by observing the videos recorded in the 
class. To see the indicators, refer to the overview of the analysis grid in the 
appendix. 

Here we limit ourselves to specifying that for each phase of a valuation 
practice, the grid has created subcategories containing the individual 
indicators. For example, during the administration and feedback phases 
the actions were grouped into subcategories referring to the use of 
different types of tests (written and oral). Researchers, moreover, have 
never considered the grid as “complete” and “definitive” as observable and 
measurable actions as indicators of an assessment practice may be much 
more numerous. 

It should be noted that grid construction work was perhaps one of the 
longest and most complex of the research carried out by the researchers 
throughout the FAMT & L project. Much support at this phase was given 
by teachers who actively collaborated on research. They helped us during 
different training and sharing moments that allowed us to observe the 
teaching practices more closely. 

Using the analysis grid tool, the research team was able to conduct 
a systematic observation study on a large sample of video sequences of 
teachers gathered in the five partner countries involved. 

The collected videos consist of real classroom situations recorded while 
implementing assessment practices such as giving a test or a task to 
students; conducting a written, oral or practical test; reflecting on the 
mistakes that have been made in a test; correcting an assigned task (in a 
group, individually or in pairs); the teacher’s answers during work on an 
individual exercise, and so on. 

From “long” videos, some short video sequences have been obtained 
that have become the main training tool for the pilot course. 
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7. Collecting videos in a web repository 


The videos collected and analysed using the grid we described also 
required the researchers to design and build a virtual environment so that 
they could be stored and categorized. For this reason, a web repository has 
been designed, according to the common model described (see paragraphs 
3 and 5 and chapter 7) to be used during the teacher training paths. 

Using the video analysis grid has therefore allowed a meta-dating of 
every short video sequence and a video recording system that facilitated 
their archiving. This archiving system has also been designed in a way 
that finding specific materials in the repository would be easy so that they 
could be integrated into “pilot” courses aimed at promoting the correct use 
of FA as a tool to improve math teaching. 

In the design phase of the repository, the partners had thought of a 
virtual environment that could contain many useful teaching objects and 
tools and have some specific characteristics. The repository needed to: 

e contain various types of teaching tools and research tools used and 
produced during the project; 

e have at least one tool that could be used for exchanges of information 
and communication (e.g. a blog); 

e be administered by all partners; for this reason English has been chosen 
as the official language, but it is possible to upload materials and 
content in all the languages of the different partners; 

e be organized in different sections based on criteria defined together by 
the partners (e.g. nationality, topics, age, etc.). 

Lastly, all the materials had to be easily accessible, usable (e.g. to 
facilitate searching for different materials) and had to remain available (e.g. 
for teachers from different countries) beyond the end of the project. 

To design and implement this kind of environment, researchers have 
looked at different solutions and followed different stages of product 
development, which in the final version has taken on features slightly 
different from the ones initially planned. 

First of all, many different digital materials and training assessment 
tools designed and used as a methodological resource for planning and 
conducting teacher training courses have been uploaded to a specific 
e-learning platform made available by French partners (e-Space Platform). 
This platform has become the support for partner courses that they wanted 
to adopt and is available for the courses that will be implemented in the 
future. The materials that can be uploaded are several: training content, 
assessment tools (such as charts, schemas, etc.), videos (educational 
situations recordings), movie extracts with text comments or related 
activities, specific tools for formative assessment, learning objects created 
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and used for teacher training, reference bibliographies, link lists and web 
sites, glossaries, project documentation file (draft, reports, alerts, meeting 
records, articles etc.). 

Instead, the videos analysed were uploaded to the web repository that 
has features and functions described in the next section. 


The FAMT&L web repository: features and functions 

The main language is English but the website is organized in 5 sections, 
one for each partner’s language: Italian (for UNIBO and SUPSD), French (for 
UCP), Dutch (for Netherlands’ Hogeschool Inholland) and Greek (for UCY). 

FAMT&L web repository hosts two different type of contents: Training 
and Context Unit. 

The Training Unit is a set of information related to a video analysed with 
the analysis grid. It shows a situation in classroom and the related analysis. 

It also displays a set of further data: last modified data, related video, 
documents that explain and expand the situation. 

Training unit are searchable in home page through facets system that 
allow to select a number of filter like partner or abilities and so on. 

A group of training unit that describes a more complex problem or 
situation are grouped in a Context unit. 

Context Unit is the second form of content type. It spokes about context 
in which one or more training unit are developed, group them together and 
show more data about it. 

Some example of data is: short description of the media, creation date, 
school level target, training unit collected, language and country. 

Context unit are listed in Context unit page. 

Over the core (training unit and context unit), web repository exposes 
some static page in which the whole project in presented and explained. 

The repository is divided in a front office, in which everyone registered 
to the site can explore, search and navigate units, contents, page and a 
back office in which certain role can manage units and contents. The back 
office allows adding, editing and removing every type of content through 
dedicated forms. 

On the top of the web site there is an edit menu from which each partner 
can manage its contents but it is necessary to have an administration role 
that permits to edit all the contents without property limitations (in the 
repo are defined, at the moment, the follow roles: register user, editor and 
administrator, each of which has specific permission and abilities to read/ 
write certain type of content). 

All the repositories are translated into four languages: English, Dutch, 
Italian, Greek, and French. So each type content is submitted in one or 
more of these languages. 
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Regarding the site interface, the home page contains a brief 

introduction to FAMT&L project and some link to search video: 

e by Context or Training Unit (tab under the logo); 

e by the 5 languages in which partners had uploaded files; 

e by author of the uploading (the 5 Partners); 

e by some section of the analysis grid (Contents, capabilities, time of 
assessment, tools or tasks to assess the students). 

In the first page it’s also possible explore directly among the list of 
video uploaded. 

The screen of the single unit (see Figure 3), however, is characterized 
by having, at the top, the video’s identification name (named according 
to the shared criteria that allows immediately to understand provenance, 
grade, school, recording year). 

In the left part of the screen is the video frame and below a brief 
description of what it contains. In some cases you can also find materials 
used by the teacher for the assessment (correction grids, forms to be filled, 
observation grid). 

On the right side of the screen there are all indicators of the analysis 
grid observable in the video: the individual actions that the teacher and the 
pupils do at different stages of an assessment process. 


[ Ele Modifica Visualizza Cronologia Segnalibri Strumenti AO 
à rameu mesa Repository x WE 


€) © tamsceduunbost/en/search-training-unit 


FAMT&L Media repository 


Home About FAMT&L Context units Training units 


A brief introduction to FAMT&L 


rocesses concerning mathematics and sciences are a fundamental component of 
inary to most of the competences that are significant in life and necessary for 


ers' and teachers’ engagement, the crisis in mathematics learning is 
becoming very diffused and serious. è EMnvkä 

The main goal of this project consists in realising a training model (using e-learning) for middle school 

math teachers (that can be applied to In-service and pre-service training). This training model (or Filter by author: 
methodology) should improve teachers’ competences: 

on educational planning and assessment (both formative and summative assessment; assessment for» UNIBO 

learning) «UO 

— on mathematics didactics. 


This set of competences would allow teachers to apply and plan specific methodologies and tools for a 


Filter by 1a. main contents: 
~ making hypotheses and pla 


id offer remedials for an ef 


ti 10 give to teachers the possibility to 


mativ gi 
je and improve their methods and allowing students to self assess and become conscious 
of their own learning processes. 


Explore training unit 


Use side filters to search and explore training units by its dimensions 


Fig. 1 - Homepage screenshot 
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È FAMT&L Media Repository x 
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learning processes based on problem solving. 


The training model is also aimed at making teachers acquire a scientific attitude consisting In: 


- analysing complex educational situations; 


~ identifying students learning needs, concerning mathematic discipline as well as transversal 


competencies (meta-competencies, learning strategies, study methods); 
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Il filmato mostra l'ultima fase del lavoro di gruppo finalizzato alla 
valutazione delle competenze argomentative e comunicative degli studenti. 
In questa fase il docente da un primo giudizio sul lavoro svolto, dicendo che 
La modalità di lavoro ha funzionato. Durante questa fase agli alunni è stata 
restituita la scheda corretta dai compagni e quindi Il dialogo si concentra 
sull'argomentazione dellslunno che difende la propria posizione, spiegando 
perché ritiene di aver risposto in maniera corretta ed esaustiva e i compagni 
che hanno fatto la correzione, i quali devono giustificare la propria 
correzione. Il docente indirizza e guida la discussione ed evidenzia un caso 


importante in cui una dimostrazione può avvalersi anche dei casi particolari: 


il caso in cui si esauriscono tutti i casi possibili. Successivamente invita una 
studentessa a cercare il modo di incorporare tutte le informazioni che ha 
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inoltre, 
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passivo. Questa situazione richiede l'intervento frequente del docente 
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1d. Secondary capabilities: Communication 


Fig. 3 - Screenshot of a Training unit 
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Following there are also other information grid that allow to understand 
what content and what skills are object of the specific assessment practice. 

The important repository functionality is the possibility to select 
through the individual voices of the grid all Training Units that contain 
videos in which the same indicators were identified. This, for example, is 
useful for searching, by a teacher, starting from the mathematical content 
or skill to be assessed. 


8. How to use of videos collected during teacher training 


In our research project we considered the international scientific debate 
(Perrenoud, 2002; Anderson, 2004; Darling-Hammond & Bransford, 2007; 
Koster & Dengerink, 2008) on teachers’ professionalism. In particular 
we know that teacher training has to be considered a strategic factor 
to improve the national educational systems (see Mumby ef al., 2002; 
Richardson & Placier, 2002; Darling-Hammond, 2006; Darling-Hammond 
et al., 2007; Coggi, 2014). 

A good part of the scientific debate about teachers’ formation activities 
seems to focus on the relationship between theory and praxis, between 
knowledge and competences, i.e. by the research of how to form the 
teachers in such a way to get that the information they gain will really 
develop into new behaviours and competences that will enter into play in 
their everyday teaching practices. 

In this line of thought, it is particularly relevant the concept of 
recurrence between theory and praxis, meaning an alternation between 
distinct (but at the same time interrelated) steps in a specific learning 
process (Atelet, 2003) which are able to translate theoretic knowledge 
and methodology into an action and also, at the same time, reflection on 
the action itself, a reflection that, in turn, becomes new knowledge, and 
so forth. There are several different contributes to this debate, based on 
interdisciplinary studies (in Pedagogy, Psychology, in Neuroscience). 

Many of those studies seem to validate the idea that a fundamental 
step for the professionalization of teachers is the identification of the most 
suitable ways to conceptualize their explicit practices in teaching (Rossi, 
2014) by means of recursive processes, integrated and interdependent 
among them (Seidel & Stiirmer, 2014), as are observation, comprehension, 
anticipation or prediction (Rivoltella, 2014) of what happens and can 
happen after a specific action. 

From here several indications stem about the most effective 
methodologies to promote the co-presence of theory and praxis in the 
formation of teachers (both in-service or pre-service). Among these, there 
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are many techniques that can be based on the use of specific support tools, 

as, in particular, the videos. 

Usually the formative activities which are based on the use of videos 
are defined “video education”, an expression which covers an ample range 
of teaching experiences, starting with the first movies in the last century, to 
the use of television and analogue supports (VHS) and then to digital (CD/ 
DVD) and ICT ones (PC and multimedia) to end up with the Internet and 
the so-called Web 2.0 (O’Reilly, 2005). 

The presence of videos in formation activities for teachers is more and 
more common, with several modalities in there use (Masats & Dooly, 2011): 
e as both an object and a tool for observation and analysis, to show a 

subject to the teachers (we speak of video-viewing, in this case); 

e as an example or display, when the video shows the practices and 
the behaviour of experienced teachers in specific situations (video 
modelling); 

e as a record of the teachers themselves, which is shared with the others, 
making it an occasion of comparison and debate with colleagues or 
with a trainer (video coaching). 

These modalities yield several implications. The videos’ content can be 
quite different: a teacher record her/him-self, use recording of colleagues or 
other experts, focus on specific didactic practices or behaviours, attitudes, 
interactions. Moreover, the videos can be presented as an example of 
everyday teaching activity (Carbonneau and Hétu, 2006; Clarke ef al., 
2008), or as a “best practice” which rarely could be directly observed 
(Seago, 2004), or as a specific experience or experimentation (Santagata & 
Guarino, 2011). Also the length of the proposed video sequences can vary, 
from very short excerpts to longer and complex sequences. 

Several studies, anyway, confirm the effectiveness of video-based 
interventions in the training of teachers: videos become a tool which is able 
to integrate and support, via the visual activity, the direct observation and 
the learning of good teaching practices of which, otherwise, there could 
only be a description, oral or written (Santagata, Zannoni, Stigler, 2007). 

In the last years the didactic technique of microteaching has gained 
much credit; actually it is a technique that dates back to the experiences 
in the ’60-’70’s by K. Romney and D. Allen at Stanford University. Allen 
himself defines (1975) microteaching as a technique which consists mainly 
in having the trainee teacher to present to a small group of students a short 
time teaching session, focused on a specific subject. The short session is 
monitored from trainers which use video recording as main tool. This will 
allow the supervisors of the microteaching session to show to the trainees, 
via the analysis of the teaching sequence, which abilities will help them 
to solve the problem in their teaching practice end the errors they can do 
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in their activities. Such an analysis can promote and facilitate a reflexion 
on what is done in the class, which contributes to an improvement of the 
teaching practices. This attention to the reflexivity as an attitude of the 
teachers to analyse and think over about their own practices, is essential 
to get an educational success (Dewey, 1961), and is what allows us to 
speak of the teachers as reflexive professionals (Schôn, 2006; Damiano, 

2007), and of a professional knowledge of their own (Calvani, Bonaiuti, 

Andreocci, 2011). 

Thus it is impossible not to see how effective the use of videos can be 
in the formation of teachers, but it is also important that this use take place 
within a well-structured formation proposal, characterized by: 

e a clear and thought over choice of the learning objectives that one 
wants to achieve with the trainees teachers (Blomberg et al., 2013; 
Seidel et al., 2011; Rossi, 2015); 

e the production or selection of the videos best suited to the defined 
objectives; 

e a good support and guide to the vision, comprehension and analysis of 
the video; 

e elaborating suitable tools for evaluation, appropriate to the objectives 
(Calvani et al., 2014). 

Following these ideas, the FAMT&L project elaborated a pilot model 
of a course for mathematics teachers that integrated and use the analysis 
of videos made in class with teachers involved in the project with 
different modalities, but all oriented to the achievement of specific 
formative targets. 

As we have already said, the idea that guided the recording-analysis of 
the videos was to be able to use the analysed video-sequences as part of 
training courses for in service teachers that can acquire specific skills in 
the use of formative assessment as an element that improves the quality of 
teaching. 

In line with the debate on teacher training, the observation of teaching 
practices by themselves would allow changes in their behaviour and 
encourage processes of reconsideration on assessment and teaching. 

In fact, the pilot course that has been developed was centred to use the 
video sequences analysed in order to promote critical thinking of teachers 
in training. 

The model of the course had been tested and its efficiency verified with 
small group of mathematics teachers in the several partner countries, so 
that it can be proposed as a model to be adopted also in other activities, 
both for in-service or pre-service teachers. 

In the following chapter we can see the more detailed description of 
these course and the main results we obtained. 
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7. Teacher training through the video-analysis: 
a model for five pilot courses. 
Mathematics Teachers Change in FA: 
monitoring of the pilot courses 


by Giorgio Bolondi, Federica Ferretti* 


1. Introduction: the frame and the objectives of the pilot 
courses 


The central phase of the project consisted in the design of the pilot 
course, which was framed on the theoretical framework shared by all the 
partners, and has been designed relying on the results obtained in the 
questionnaires administered to teachers and students at the beginning of 
the project. 

The theoretical-methodological shared framework used for the design 
of the pilot course defines the key points of formative assessment in 
mathematics and the pilot course was intended to answer to the teachers” 
formative needs pointed out by the questionnaires. The main purpose was 
to focus on the basic elements of the observation of classroom practices 
and, in particular, assessment practices, in order to capture those peculiar 
aspects that have to do with its formative function in mathematics. 

In according to our FAMT&L theoretical framework we started from 
the premise of considering assessment as assessment for learning and not 
assessment of learning and assuming that the implementation of formative 
assessment leads to higher learning quality (Wiliam, Lee, Harrison & 
Black, 2004). 

The main purpose of the course was to build good assessment skills for 
math teachers of secondary schools (grade 6 to 10), trying to significantly 
affect their knowledge, skills, by working also on metacognitive aspects. 

The objective of improving assessment skills, in the direction of 
specific and consolidated knowledge, skills and beliefs of teachers, made 
the frame of each activity carried out during the training course. In 
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particular, the work done in classroom about the beliefs of each trainer was 
crucial, both in large group discussions and in small group activities, with 
the aim of developing a shared learning culture of assessment for learning 
in mathematics. 

From a methodological point of view, the central idea was to implement 
video analysis techniques, as a key tool for teacher training. Therefore, 
the web repository constituted the central tool of the entire path, based on 
recursive processes of video analysis, individual reflection, discussions in 
large and small group. As we shall see in the next paragraph, the importance 
of video analysis in the field of teacher training is now shared at the 
international level; video analysis in fact plays a fundamental role to act as 
teachers’ analytical skills and the consequent development of their reflective 
thinking (Santagata and Angelici, 2010; Santagata and Guarino, 2011). 


2. The importance of formative assessment in mathematics 
in teacher’s professional development 


We recall here some theoretical aspects that have guided and served 
as the frame of reference for all research activities: the role of formative 
assessment in mathematics. 

Looking for a generic (and etymological) definition of the term 
evaluation, we could first say that it consists in “attributing value to 
something or someone”. To attribute this value, those who have an 
assessment responsibility should follow coherently a well-defined 
assessment-related model, and rely on rigorously designed techniques and 
tools. Following the latest assessment theories, the role of evaluation can 
thus be essential to ensure a quality education and training system with 
respect to the level of skills of pupils (Vannini, 2009). In a peculiar way, 
the role that the evaluation should have is the formative function (the term 
being originally coined by Scriven, 1967). 

Formative assessment should in fact be an indispensable part at all 
stages of the teaching-learning process. In this regard, its main function 
is regulatory as it aims to modulate and adapt the training process 
continuously, especially if this is oriented to the individualization of 
teaching-learning steps (Tornar, 2001). We can then define the assessment 
as collecting evidence and information about the teaching-learning process, 
thus leading to a change of perspective also in mathematics teaching, from 
the assessment of learning to assessment for learning. In this framework, 
therefore, the concept of formative assessment in mathematics becomes 
ever more important. In school practice, assessment in mathematics is 
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often structured and organized according to the mathematical content, but 
the education mathematics community is increasingly questioning if this 1s 
the optimal way, in a formative perspective. We can found a great support 
in this direction in Martha Isabel Fandifio Pinilla’s research results, in 
which the different components of mathematics learning are analyzed in 
depth. The frame there constructed is very useful for defining a dimension, 
other than the content one, for assessment (Fandiño Pinilla, 2008). As we 
read in the author’s words, mathematics is one and learning mathematics is 
a unitary fact; it is however useful to distinguish several components of it, 
which are: 

e conceptual learning; 

e algorithmic learning; 

e strategic learning; 

e semiotic learning; 

e communicative learning. 

These reflections have a strong impact on the assessment, because 
in order to carry out a detailed analysis of the learning, to be used 
for intervening on the learning process, it is necessary to treat these 
components both independently and in their interplay. The overall study of 
components of math learning is so useful in investigating what works and 
what does not work throughout the math teaching-learning process. This 
theoretical-methodological framework, moreover, helps to investigate the 
causes of student errors, a fundamental process in a formative assessment. 
It is precisely with reference to the above mentioned theoretical framework 
that the pilot course has been developed. 


3. The design and the methodology of the pilot courses 


The core of the pilot courses consisted in activities aimed at 
explicating teachers’ beliefs, by sharing of beliefs and opinions about 
formative assessment in a small and large group. As we know, this 
initial shift from the analysis of personal beliefs to the building of a 
“useful learning” collective idea is strongly influenced by the complex 
social interactions that come into being within social groups (Hoyles, 
1992). Particular attention was paid to these moments, in particular 
by stimulating the comparison between teachers ‘analysis of the video 
proposed in the web repository. For the design of the pilot courses, we 
worked on a collaborative approach classroom situation and on the idea of 
individualization, diversifying classroom interventions also in according to 
individual teachers’ needs. At all phases of the course, however, we have 
tried to pursue — for all the participants — some fundamental objectives: 
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e to identify and recognize, among other things, situations of formative 
assessment in the classroom; 

e to analyze the details, and identify non-functional evaluation practices; 

e to reflect on their own way of evaluating in the classroom and 
identifying what analogies and differences with the situations being 
analyzed. 

From the point of view of the didactic methodologies adopted during 
the course, we adopted a constant collaborative work approach between 
participants and researchers (Morissette & Desgané, 2009); we have 
therefore tried to make the participants active during each classroom 
sessions, to achieve a shared knowledge building. Participants have 
been considered as competent actors who have the resources to act, 
reflect and theorize on their actions. Discussions with experts and group 
comparisons were the strategies with which the acquisition of knowledge 
and abilities and the change of individual beliefs were promoted most, 
starting from the need to analyze and then internalize what is gradually 
was shared. 


4. The structure of pilot course 


Pilot training models have been personalized by each partner country, 
based on individual resources and learning needs. However, there have 
been points in common both as regards the definition of aims, objectives 
and content, as well as the methodologies and assessment practices. 

The training model will then be used to define training paths that 
promote the correct use of formative assessment in mathematics training 
assessment by encouraging critical and reflective thinking of participants 
by analyzing effective/ineffective strategies through video analysis tools 
and techniques. 

The aim was therefore to create a training model on which to build 
training paths, both in presence and in blended learning mode. In Annex 
1, we show the structure and the organization of model training; the 
following table (Tab. 1) shows the structure and dates of each partner 
country’s pilot courses (FAMT&L — Final Report WP4). 
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Tab. 1 - The structures and the organizations of models training 


Italy Switzerlan Cyprus Holland France 
When 5 meetings 2 meetings Two courses 4 meetings 6 hours of 
7 october 2016 each 6 hours each 1,5 hours training 
(5 hour) 7 march 2016 3 meetings 10 June 2016 course begin 
14 october 18 april 2016 each 4 hours 24 June 2016 10 october 
2016 (5 hour) 7, 8, 9 June (via the 
15 october 2016 Next school E-space and 
2016 (8 hours) year: OpenEdx 
28 october 3 meetings 4 September platform). 
2016 (5 hours) each 4 hours 2016 
29 october 4, 5, 6 July 9 October 2016 
2016 (7 hours) 2016 
Afterwards 
periodic 
meetings in 
Professional 
Learning 
Community 
(every 6 weeks, 
each 1 hour) 
until end of 
year. This 
means training 
will continue 
after FAMT&L 
has ended. 
Number 5 4 3 2 This is a Mooc 
of online (via the 
trainers platforms so 
there will be no 
trainers). 
Function + Two trainers + Two trainers + Preparation e Mathematics The Mooc is 
of with with of material. (didactics) created by 
trainers expertise on expertise on + Organizing specialist a teacher of 
the subject the subject and and Maths. 
of evaluation of evaluation coordinating specialist 
and three and two the meetings in formative 
teachers of teachers of with assessment 
mathematics mathematics teachers. e Pedagogy 
education. education. trainer, 
e Preparation e Preparation specialist in 
of material. of material. assessment 
e Organizing e Organizing 
and and 
coordinating coordinating 
the meetings the meetings 
with with 
teachers. teachers. 
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Tab. 1 - continued 


Italy Switzerlan Cyprus Holland France 
Number 15-20 18-24 Group 1: 16 18 This Mooc will 
of post-graduate be opened 
teacher teachers to the whole 
(kindergarten/ education 
primary/ institute (ESPE), 
secondary). every students 
(teachers part 
Group 2: 10 time and also 
secondary full time) could 
mathematics participate at 
teachers. this Mooc 
3500 students 
of ESPE of 
Versailles. 
Which Mainly High Teachers in Two In-service Teachers in 
kind of and Low training who possibilities: teachers training who 
teacher Secondary teach part time 1. Post- teach part time 
Teachers. as teachers in a graduate Training is and full time as 
class of middle teachers validated by teachers ina 
school. (master nationwide class of middle 
students in registry www. school. 
Mathematics registerleraar. 
Education). nl (validation 
2. In service number 
mathematics wpZs7owYcw). 
teachers. Teachers 
will receive 
certificate to 
prove their 
participation 
(mandatory 
in-service 
training which 
is registered for 
every teacher). 
Number 30 hours 2 meetings 12 hours: 3 In total 12 6 hours 
of hours of 3h meetings of 4h in course, 
thereafter 6 
hours in PLG- 
meetings. 
Use the Use of platform We will not use We will not use We will use (Via the 
e-space mainly for the e-space the e-space videos available E-space and 
uploading/ platform, but platform. on internet OpenEdx 
using material. we do face-to- We use the (teaching platform). 
face lessons. web-repository  channel.org). 
We will use for watching E-space 
the repository the videos platform not 
and the We mainly work used. 


questionnaire 
on-line start 
and end. 


face to face. 
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As we see in the table above, the pilot course implementation modes 
were different in each country, but the common structure nevertheless 
allowed a constructive comparison. As we read in Annex 1, certain 
steps are foreseen for each modality of realization of course, so that all 
courses are consistent with the planned design in line with the theoretical 
framework. For instance, some countries, like France, have been the 
course completely online on E-space environment, while others, such as 
Switzerland and the Netherlands, have not used the E-space; this difference 
in achievement has allowed highlighting the strengths and weaknesses 
of each modality of course implementation. As we will see in the next 
paragraph, the web repository and the FAMT&L grid have been a common 
video analysis tools for all pilot courses and they have been fundamentals 
for delivering common goals. 


41. The crucial roles of the FAMT&L Grid and of the Web 
repository within the pilot courses 


The Web repository of the project consists of recorded videos 
containing formative assessment situations in the mathematical classroom, 
carried out by associated partner schools of the project itself. With 
videos collected and analyzed, FAMT&L researchers have created a 
web-repository that contains all the videos, hundreds of microsequences 
and their video analyzes. In particular each country has produced “long 
videos” in which there is a total recorded lesson and from every long 
video has been obtained some microsequences that portray training 
assessment situations; within the web repository it is possible to trace each 
microsequence on long video. 

These videos have been analyzed and indexed according to 
a structured grid, the FAMT&L Grid (Annex 1), designed to analyze 
mathematics formative assessment practices. As we can see in Chapter 4, 
the FAMT&L Grid was developed in line with the formative assessment 
theoretical framework shared and, starting from it, quality indicators for 
formative assessment mathematics teachers’ practices were described. 
The indicators (validated by the researchers after several trials within 
each country) were assembled by different categories, related to the 
specific phases of the formative assessment: e.g., sharing of assessment 
criteria, the administration of assessment (write or oral), the formative 
use of the error, the self and peer assessment and the formative feedback. 
Each set of indicators behaves like a specific check list, to be used 
in Web repository to label the “class scene” that is taking place on 
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the video. The indicators were placed within a largest scheme that 
includes further and fundamental macro-variables: the description of 
the class setting, the timing of the evaluation practice, the mathematical 
knowledge and skills involved (based on the theoretical framework 
OECD-Pisa, 2013). 

The indicators were used in the pilot courses for recovering situations 
to compare, for opposing behaviors, for outlining categories. 

Every long video has been meta-dated in the following categories; 
moreover each Country conducts a micro-analysis of each short episode, 
with the following macro-categories (as we can see in Chapter 4). 

The web repository has been a fundamental tool in all pilot training 
courses; in each country all coursers have been provided with a temporary 
password, thus having the opportunity to watching videos freely and 
getting in touch with the video analysis FAMT&L grid. The video analysis 
grid played a key role in these training paths, having objected, at first, 
to a free exploration by the participants and then — gradually during 
the training courses — of systematic use to focus on the details of the 
various videos, both in the classroom (in large and in small group) and 
individually. 


5. First results of training model courses 


As we have seen, critical and reflective analysis datasheet, individual 
or on group, have been compiled throughout the course, and each trainee 
has consistently compiled a personal “diary on board”. The research 
tried to analyse some “changes” in the skills of teachers at the end of 
the pilot course; indicators of acquired knowledge, practical skills and 
metacognitive aspects have been identified and investigated. Regarding 
the knowledge of assessment and, specifically, formative assessment in 
mathematics, it was chosen to analyse the learning perceptions by the 
participants, that is, they monitored as they progressed through the training 
course. In particular, it was attempted to investigate how their knowledge 
of the theoretical reference framework, their ability to recognize and 
analyse classroom assessment situations, to argue the effectiveness of this 
formative function. 

In general, by analysing the documents compiled by the trainees 
throughout all the course implemented within the project, one can 
conclude that there has been an increase in awareness of both the 
theoretical framework of mathematical formative assessment and in the 
ability to analyse (and self-analyse) the classroom situations. 
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5.1. The Italian research 


For instance, the Italian research group carried out a pre-experimental 
design to investigate the above-mentioned aspects. In details, during the 
Italian pilot course, three main detection tools were used (almost all the 
questionnaires are the same of the questionnaires administered in the other 
countries pilot course): 

e a questionnaire on “learning perceptions”, which has highlighted the 
perception of knowledge and skills of the participant; 

e a questionnaire on educational beliefs, structured with two item 
batteries with 4-level of agreement, from “a lot” to “nothing”. The first 
battery concerns a Likert scale on the “school idea” (see Vannini, 2009) 
that contains a sub-scale on the ideology of “students’ natural skills” 
and, on the other hand, another sub-scale on teachers’ confidence in 
the potential of teaching in terms of learning opportunities. The second 
battery concerns the beliefs on the formative function of the evaluation, 
in line with FAMT&L shared definition; 

e finally, the Norwegian Teacher Self-Efficacy Scale (NTSES, Skaalvik 
& Skaalvik, 2009) validated in the Italian context (Avanzi et al., 2013), 
composed of 24 items divided into 6 dimensions, to which the teacher 
must respond by declaring “if you feel able to...” implement specific 
behaviors in defined school situations, with a gradation form “nothing” 
to “totally” responses). 

As we can read in Ferretti & Vannini (2017), research has brought good 
results in almost every direction; below we show the most relevant results. 


Learning — Knowledge and Abilities perception 
In the questionnaire on learning perceptions, participants were asked to 
score a score from 1 to 5 (low to high) on their perception of competence 
compared to each of the following indicators: 
e “knowledge of the theoretical framework on formative assessment”; 
e “explain the reason why formative assessment is a high rate of 
effectiveness”; 
e “being able to analyse a classroom assessment situation with 
predetermined indicators”; 
e “know how to organize in the classroom a situation where the formative 
assessment is integrated in the didactic design”. 
The questionnaire was administered at the beginning, during and at the 
end of course. 
With reference to all indicators, there is an increase in the average 
scores of responses provided at the end of the training course compared 
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to the intermediate and initial training. The trend is increasing in all four 
cases; there is no inversion and no constant trend. The T-test for paired 
samples shows that all differences are significant at 0.0001. 

Although there is a significant increase in perception of competence 
and this represents an important indicator of the effectiveness of 
intervention, perceptions do not show real mastery of skills, knowledge 
and more generally of competences. Particularly relevant data is the almost 
linear increase in the scores with respect to the indicator “To be able to 
analyse a classroom situation with predetermined indicators”, indicating 
a significant increase in the perceptions of the participants of their ability 
to analyse video situations. It is shared internationally that the reflexive 
skill of teachers grows as they refine their ability to analyse details in 
videos and to think and discuss about video situations. The increase in 
critical reflexive skills and in analytical thinking of the teachers was also 
perceived during the working group. Another one interesting information 
is the increase in the scores obtained in the indicator that refers to the 
perception of one’s own skills of “Know how to organize a classroom 
where the formative assessment is integrated in the didactic design”. 
The average referring the scores obtained has increased by 1.5 points, 
indicating how the pilot course had a positive impact on the perceptions of 
the trainees on their “practical” assessment skills. 

These results, although entirely “exploratory” and exclusively linked to 
self-perceptions of competence, are particularly relevant as the expectation 
of teacher practices change is the most important goal to be pursued. 


Self-Efficacy Scale 

Regarding the Self-Efficacy questionnaire, we have process aggregate 
data for the 5 sub-scales, comparing the answers provided by participants 
at the beginning and at the end of the pilot course. The scores of almost 
all sub-scales have increased significantly and the related T-test for paired 
samples gave results of significance below 0.05. This sensible increase 
in sense of Self-Efficacy in almost all dimensions investigated is totally 
in line with the perception of improving the ability of the teachers to put 
into practices and techniques of formative assessment and with various 
international studies that hypothesizes the positive correlation between the 
use of formative assessment in class and the Self-Efficacy of the teacher. 


6. Conclusions 


The analysed results of the Italian group’s research are in line with the 
results obtained in other countries’ pilot courses (some of which are still 
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being analysed). The results are of particular utility for the development of 
more clear hypotheses on the teachers’ professional development courses 
and on the use of video analysis methodology. 

The research team has verify that video analysis seems to reveal its 
potential only if it is integrated into a well-structured teacher training 
proposal. One of the objectives of the FAMT&L research group is to 
carefully analyse all the data obtained in the various pilot courses, 
compare and study them. 

An in-depth analysis of all the results will enable the development of 
well-structured pilot courses in terms of: 

e a clear and reasoned choice of learning objectives to be achieved with 
the group of teachers in training; 

e the production or selection of videos that best fit the goals to be 
achieved; 

e the elaboration of appropriate reflexive and self-evaluation tools, also 
aligned with the objectives. 
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8. The voice of the teachers: 
their experience in FAMT&L 


by Andrea Maffia*, Federica Recchiuti*, Camilla Spagnolo* 


1. Introduction 


Pre-service teachers’ education usually consists in a theoretical part, 
then followed by some kind of practice in school. In our experience, it is 
quite common to hear about the discrepancy between what is taught in 
universities and what is put in practice. If you are the new young teacher 
of your school, doing things differently than your more experienced and 
more estimated colleagues can make you feeling uncomfortably. In this 
kind of situation two different paths are walkable: you adjust your practices 
to those of others, or you try to strengthen your own beliefs about teaching. 
Participating to FAMT&L has been for us the chance to follow the second 
way. Attending further teacher education, comparing our practices with 
those of other teachers and observing ourselves were opportunities to 
criticize what we do as teacher and so to acquire new awareness on our job. 

In this chapter we highlight how FAMT&L involved us — in 
different ways — as mathematics teachers and how it triggered a process 
of pondering on our formative assessment practices. Each one of the 
following sections concerns a particular aspect of the involvement of each 
one of us. In particular, the first section is written by the first author, the 
second section is by the third author and the last section by the second one. 


2. Cameras in the maths class: videotaping your own 
teaching 


One of the main features of the FAMT&L project is the video database 
for formative assessment practices. Obviously, the realization of such a 
repository involved mathematics teachers and their classes. 


* Italian Secondary School Teachers. 
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Videotaping of a lesson does not seem so difficult for a teacher: you 
just should do what you usually do while someone stays there behind a 
camera. There is nothing more. However, when you are really asked to 
be videotaped, several worries come out. The origin of those worries 
may be the idea of someone else looking at that video while thinking 
that you are a good teacher or a bad one. All the mistakes you do during 
the recorded class will be permanently impressed in the film (or better a 
digital memory) and the others will be able to see them. 

Usually, what you do and say in your own class remains between you 
and your students. The idea of someone watching at you while you are 
working brings you back to the feeling of being judged (or maybe assessed, 
or evaluated?). In the exact moment in which you realize it, a process of 
rethinking on how you evaluate your students’ work begins. Videos are 
used in FAMT&L to discuss and to rethink formative assessment practices. 
This means that watching at those videos will not result in a judgment of 
the involved teacher but in a formative process of development of practices. 
This means that if any assessment is made through those videos, it is 
formative assessment. 

Feeling part of a formative assessment process (rather than a summative 
one) is definitely less stressing: Isn’t it the same for students? Even if, as 
a teacher, you are alone while the video operator is filming you, is quite 
better when you also know that you are part of a project. In the project, 
there are some other teachers that are videotaped as you are. In some 
sense, you become part of a potential community of practice, in the sense 
of Wenger (1998). Finally, it seems a bit less risky. 

Once I accepted to be videotaped, then I informed students’ parents 
and colleagues about the event of a video operator coming to school. 
One of the most common question that I got back was “how can it be 
spontaneous?”. The main concerns were about the fact that pupils could 
be scared or inattentive; colleagues asked about my teaching: would it be 
the same without a camera recording? The experience within FAMT&L 
gives me an anecdotic knowledge to give some possible answers to these 
questions. 

When the red light of the camera is turned on, some students notice 
it. It is easy to realize it both during the class, both watching the video: 
their gazes are taken by the camera. Then the class starts, it is time for 
doing mathematics, their attention is suddenly oriented to the tasks. 
They do some peer-to-peer work, the teacher walks around the room 
to hear what is happening and it is easy to see that the dynamics are 
the same as always. Apparently, nothing is different from the student’s 
viewpoint. 
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What about the teacher? Thinking back to the moment of videotaping, 
I can assure that there was not a moment in which the main concern was 
the camera or the video operator. When the class is going on there are 
continuous stimuli by pupils. There is not enough time to consider both 
the camera and the lesson. If we just give a look to the video, it seems that 
also from the teacher viewpoint everything is as it has always been. But, to 
be honest, it is not exactly the case: the main difference is not visible in the 
classroom because it was exploited before. 

As a matter of fact, being somehow evaluated prompts to put a 
considerable attention in the designing of the videotaped class. You try to 
make as clear as possible your aims for that lesson; you chose carefully 
the tasks and the way of presenting them; you ponder on the groups of 
students working together; you redesign your assessment rubric. Going 
back to former readings (books, university notes, papers, ...) to see 
if you considered everything you were taught, you begin a process of 
critique to the didactical tools you already use in your classes and you 
rethink them and their use. Finally, the implemented class is not the 
same as it would be without the camera, but it is not because of the 
video operator in the room; the difference depends on a reflection that 
took place before. 

When the camera is turned off, it is not still done. Before everybody 
can see that video, you want to give it a watch. As far as you put a 
strong effort in designing the lesson, you can see clearly everything that 
is not as it was in the plans. Watching the video several questions come 
out: questions about how things went and why they went that way. The 
powerful opportunity given by the video is to watch it a second time — 
maybe a third — to try to understand it. A process of self-confrontation 
(Fuller & Manning, 1973) begins. 

Videos in FAMT&L repository have also been subtitled and so a new 
opportunity rises. Reading back the dialogues allows putting attention to 
the words that were used, both by the teacher and the students. So far, 
most of the rethinking was about teacher’s practices: a slow reading of 
what students said switch the focus on their learning. Why did they use 
those words? Why were they so secure on some topics? What is the source 
of a particular difficulty I can notice? 

Even if it is thought as a tool for other teachers’ training, the video 
itself becomes for the teacher a tool for formative assessment of his 
students (Ferretti & Maffia, 2017). The richness in thinking and reflection 
that can be evidenced along all this process made me feel the necessity of 
videotaping some other classes in the future; it can be useful for others, it 
is very fruitful for me. 
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3. The experience of the pilot training course: learning 
from videos 


The assessment in mathematics has a strong impact on students and 
teachers alike. Moreover, many researches have shown the fact that the 
assessment is the most important factor in determining the positive or 
negative behaviour of students when approaching maths. For this reason, a 
problem of crucial importance is how the teachers evaluate their students’ 
learning. 

This problem has always been there for a teacher, but it rises even 
earlier when you are studying to become a teacher. The first approach that 
I had with the FAMT&L project was during my graduation, while I was 
writing my thesis (Spagnolo, 2016). I will later describe my experience as a 
teacher in training. 

During our course of study, we often asked ourselves if what we learn 
from the textbooks or in the classes is enough to safely step into the 
classroom. Will we be able to deal with the variables that show up in 
class during an explanation, a work group or an open discussion? The web 
repository that was implemented during the realization of the FAMT&L 
project includes a grand total of 126 videos of situations that actually took 
place in the classroom, that allow teachers still in training to perceive 
the life of the classroom without having experienced it first-hand, and to 
observe with a critical eye every detail within the context of the class. 
Of these 126 videos, 91 are in Italian and they record the work of 58 
teachers from Italy and 69 from the Canton Ticino (Switzerland). It is 
also possible to watch videos in every language, thanks to the subtitles 
that each country provided both in their mother tongue and in English. 
The collaboration among the project partners allowed developing a grid of 
punctual indicators for each phase of the formative assessment. This grid 
was used during the training course to analyse in detail some of the video 
sequences of classroom situations. 

This research project focuses on training the teachers on formative 
assessment. The objective is to help the teachers to learn how to use 
formative assessment in mathematics (Ferretti & Lovece, 2015). The tool 
of working on videos about real classroom situations shapes an innovative 
way to promote a correct use of formative assessment in teaching-learning 
situations. It is not the first time that videos are used as a tool for training. 
In fact, the same tool can be found in tutorials or in videos representing 
significant moments of a class, created ad hoc, sometimes even with the 
help of actors. The real innovation is not in the use itself of the videos for 
training, but rather in the fact that they display moments of a classroom’s 
real life. 
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The pilot training course was structured in remote and presence 
activities, as well as small groups or individual activities. The workload 
was distributed during the whole course, and it allowed everyone to 
participate to the proposed activities. The remote activities consisted in 
both an individual study and the development of group activities by using 
the course platform (where many different kinds of educational material 
were available, such as learning context examples, videos of real-life 
situations in teaching maths, assessment tools, training paths, etc.) and 
the web repository for the analysis of the scenes of teachers in class. The 
work was checked and verified both in itinere, through feedback regarding 
the individual works that were made during the various phases, and at the 
end of the course, through an interview meant to comment the work done 
during the whole path of the project. 

Besides always being available on the platform, the videos were also 
checked and discussed during the training course. The classes were 
scheduled in order to alternate such moments to moments of lecture, 
discussion about the grid, group work and planning of potential classroom 
activities. The lectures had the aim of introducimg both basic notions and 
specific notions, fundamental in order for each participant to be able to 
express their point of view regarding the examples of assessment that were 
in the video sequences in the platform. These kinds of activities allowed 
sharing everyone’s opinion, everyone’s experience and methodologies 
among the course’s participants, in order to strengthen and enhance the 
teachers’ knowledge on project didactics and on formative and summative 
assessment and assessment for learning. 

As it is described in the section of the National Guidelines for the First 
Cycle dedicated to the assessment, “La valutazione precede, accompagna 
e segue i percorsi curricolari. Attiva le azioni da intraprendere, regola 
quelle avviate, promuove il bilancio critico su quelle condotte a termine. 
Assume una preminente funzione formativa, di accompagnamento dei 
processi di apprendimento e di stimolo al miglioramento continuo” 
(MIUR, p. 13)'. 

Now that I am a teacher I can take advantage of what I’ve learnt during 
the training. 


1. Trad. “Evaluation precedes, accompanies and follows the curriculum. It activates 
the actions, it regulates those already initiated, and it promotes the critical balance on 
those completed. It assumes a formative function, accompaning the learning processes and 
stimulating continuous improvement”. 
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4. Observing and using the video-analysis grid: my 
experience 


I took part to the course “La valutazione formativa per il miglioramento 
della didattica della matematica nelle scuole secondarie e nella formazione 
professionale” (formative assessment for mathematics education 
development in secondary school and vocational education) as a pre- 
service teacher. After participating to that course, I started to teach in 
school (grades 6-8). 

During classes, each participant received a video-analysis grid that was 
used to analyse videos from the web-repository. Some of these video were 
viewed during the course lessons, they were chosen among the several 
available on the platform that was open for all the course participants. 

The grid was useful to me for, at least, three reasons. First of all, 
the 126 videos in the web-repository are labelled; in this way, each user 
can choose those videos that are more pertinent to his own research 
according to his specific interest. It is possible to search specifically for 
each indicator in the grid, to look for Italian of non-Italian videos, to 
compare different class dynamics, different ways of acting — by teachers 
and students — and different didactical strategies. 

The second reason is in the usage of grid to analyse videos. Such an 
activity was often proposed during the lectures in the course. Videos 
have always been selected with a perfect adherence to the topic that was 
discussed in the work groups; so there was room for debate, ideas sharing. 
This was possible also to me, as a pre-service teacher. 

The grid is divided in five blocks, each one is about a specific aspect or 
moment of formative assessment: such division is, in my opinion, doubly 
useful: on one side it allows to focus the attention to the single evaluation 
phases in the chosen video; on the other side it offers the opportunity to 
analyse each of those phases in a itemized and detailed way. The chance 
to look at teachers and students in “real class life’ moments is a precious 
experience for those who do not still have experience as teacher in school, 
but also for those who do. Such activity gives the opportunity to interface 
with yourself and more experienced colleagues. So, using the grid, that 
moment was even more formative than expected: we had a useful tool to 
choose what to look at, a set of shared criteria to analyse videos; it is a 
concrete tool to analyse critically the proposed dynamics. 

To these positive features, I would add a third reason: the using the 
grid as an analytic observation tool in my classes, especially in some 
phases of formative assessment activities. First of all, it is useful to me to 
structure a priori the different phases of assessment: sharing of evaluation 
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criteria With students, the administering of the task, data collection, self- 
assessment or peer-evaluation, formal interactions. 

Along my experience as a teacher in a middle school, it was very 
important the presence of a tool to plan a priori such fundamental 
moments. Last, but not the least, the grid is useful to focus the attention 
to the single moment composing the different phases of formative 
assessment, especially when you evaluate your own behaviour. 
Participating to these classes, I learned that, when a lesson includes 
formative assessment, it is both an end and a beginning: after sharing the 
criteria and aims, we design a suitable task for our students (according to 
times and contents); we administer the task in such a way that students 
can stay calm and the teacher can collect as much information as possible. 
Then there is room for feedbacks. Such a moment is the beginning 
on a new path that has to be “invented” basing on the analysis of the 
information collected by the teacher at the end of a formative assessment 
activity. 


5. Conclusions 


Looking back to our experiences along the FAMT&L project, we can 
realize that each one of us had different but similar gains. As young 
mathematics teachers we felt that our training during university concerning 
the theme of formative assessment was not enough. Participating to the 
project gave us the opportunity to reflect on how important formative 
assessment is for our practice in the classroom, but we realized also that 
we did not have enough tools (or maybe enough awareness and certainty) 
to realize it significantly and effectively with our pupils. 

At the end of the project, we would not say that we feel as experts 
of this topic; however, we all feel the strong belief that we will re-invest 
what we have learnt in our future classes. In particular, the possibility 
of watching videos (about our classes or others’) and discuss about them 
together with other colleagues appears to us as a strongly formative 
experience in the sense that it was the change to form and give shape to 
new practices and/or to reshape those that were already in our classes. 

Finally, we open up some questions for us and for those others that 
were involved in the project: how can we, as teachers, promote the culture 
of formative assessment? Can we imagine exporting the model proposed 
by FAMT&L as training for future pre-service and in-service teachers? We 
would suggest that “yes” is a good answer. 
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9. Strengths and weaknesses: 
quality of the project and possibilities 
for improvement 


by Seerp de Blauw*, Rob Velder* 


1. Introduction 


To measure the effective change in teachers’ assessment practice 
participating in FAMT&L pilot training course, the European partners 
have defined a specific evaluation model of the course. 

Effective evaluation of a training is seen as an important step for 
different reasons (see paragraph 2.2). 

During the development of the training course and web-repository in 
the FAMT&L project a strategy was put in place for evaluation based on 
five steps. The fifth and final form of evaluation is the evaluation of the 
pilot training course. 

Adopting the Kirkpatrick model (Kirkpatrick, 1959) we designed a specific 
plan to evaluate and validate the course model. In particular, specific evaluation 
instruments have been designed, as some questionnaire to be administrated 
before, during and after the training path. These tools are common for all 
partners but everyone was able to choose whether to add specific questions to 
detect, for example, opinions and beliefs on specific topics of the school. 

This chapter outlines theory on evaluation and the use of critical 
friends (9.6) and the evaluation strategy defined in the FAMT&L project. 
Both literature and strategy are focussed on evaluation of professional 
development in teacher education. Hence in paragraph 2 references are 
made to literature on professional development since this has implications 
for the design of the training courses and evaluation strategy and 
outcomes. The defined evaluation strategy is described in paragraph 3. The 
role of critical friends and obtained feedback is focus point of paragraph 4. 
The fifth and final paragraph outlines the outcomes of the evaluation and 
strong and weak points of the designed pilot training course. 


* Netherlands’ Hogeschool Inholland. 


138 


Copyright © 2018 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788891774637 


2. On the theory of evaluation 
Effective evaluation begins even before a program starts: 


Trainers must begin with desired results and then determine what behaviour 
is needed to accomplish them. Then trainers must determine the attitudes, 
knowledge, and skills that are necessary to bring about the desired behaviour(s). 
The final challenge is to present the training program in a way that enables the 
participants not only to learn what they need to know but also to react favourably 
to the program (Kirkpatrick, 1993). 


It is important that the results are defined in measurable terms so 
that all involved can see the ultimate destination of the initiative. Clearly 
defined results will increase the likelihood that resources will be used most 
effectively and efficiently to accomplish the mission. 


2.1. Levels of evaluation 


Level 1 Level? Level3 Level 4 
Reaction Learning Behavior Results 


Researcher referred to Kirkpatrick model that was first introduced in 1954 
by Kirkpatrick. In 2008 an expansion of the original four levels was mad in the 
Kirkpatrick Partnership model. Finally in 2010 Jim and Wendy Kirkpatrick 
clarified the original four levels in the New World Kirkpatrick model. 


Level 1: Reaction 
To what degree participants react favourably to the training or customer 

Satisfaction. 

Engagement: The degree to which participants are actively involved in and 
contributing to the learning experience. 

Relevance: The degree to which training participants will have the 
opportunity to use or apply what they learned in training on the job. 


Level 2: Learning 

To what degree participants acquire the intended knowledge, skills, 
attitudes, confidence and commitment based on their participation in a 
training event. 
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Knowledge: “I know it”. 

Skill: “I can do it right now”. 

Attitude: “I believe this will be worthwhile to do on the job”. 
Confidence: “I think I can do it on the job”. 

Commitment: “I intend to do it on the job”. 


Level 3: Behaviour 
To what degree participants apply what they learned during training 
when they are back on the job. 
Required Drivers: Processes and systems that reinforce, encourage and 
reward performance of critical behaviours on the job. 


Level 4: Results 
To what degree targeted outcomes occur as a result of the training event 

and subsequent reinforcement. 

Leading Indicators: Short-term observations and measurements suggesting 
that critical behaviours are on track to create a positive impact on 
desired results. 


2.2. Design evaluation on training courses 


Effective evaluation of a training is seen as an important step for different 
reasons: 

e reflection ensures a higher standard of the training course because of 
structured feedback and consequent adaption of model and materials; 

e in the development of a training the direct feedback of participants, 
being the trainer and trainees, are most likely not received; 

e it can create a higher commitment from future participants. 

Before an evaluation can be done with effectively the outcomes of the 
training must be defined. The second step is the designing of evaluation 
instruments. Since the course is directed at formative assessment, or 
Assessment for learning, evaluation of the learning process is part of the 
training and can as be used for evaluation of the training course. The third 
step consist of collecting data and drawing conclusions. These conclusions 
will be used to interview the trainer and some trainees. 


2.3. The purpose of evaluation 
To begin it is important to clarify the purpose of evaluation, in term 


of process, product, outcomes, program and effectiveness evaluation. 
Following some description of each element. 
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Process evaluation 

Process evaluation is a means of implementing and refining the 
program’s design. This evaluation procedure addresses information about 
how well how well the implementation and design of the program is going 
and what, if any, obstacles conflict with the program’s success. 

The key factor in implementing this evaluation procedure is interaction 
between evaluators, designer and stakeholders. 


Product evaluation 

Product evaluation refers to ultimate decision associated with the 
fate of the program (Fitzpatrick et al., 2004). This decision may include 
modification or refocusing of the program under review (Stufflebaum, 
2003). The outcome is a product of collections of descriptions and numerous 
archived judgements about the objectives, merit and worth of the program. 

In order for the evaluator to arrive at a conclusion he must collect both 
quantitative and qualitative information from personnel and stakeholders 
involved. If necessary product evaluation can be divided into subcategories 
of impact, effectiveness, sustainability and transportability in order to gain 
more concise information about the long term effects of the program. 


Outcome-based evaluation 
In outcome-based evaluation (OBE) four types of evaluation are most 
commonly used: 
1. Program evaluation 
This uses person-referenced or organization-referenced outcomes to 
determine whether a program is meeting the desired outcomes and uses. 
2. Effectiveness evaluation 
This strives to report the extent to which a program is meeting its goals 
and objectives. 
3. Impact evaluation 
This studies whether or not a program has made difference for its 
stakeholders compared to an alternative. 
4. Policy evaluation 
This researches the equity, efficiency or effectiveness of outcomes for a 
program (Schalock, 2001). 


Program evaluation 

Program evaluation typically strives to answer the question, “What 
outcome is my program producing in its serving recipients?”. These 
generally fall into four categories: 
1. organization performance outcomes; 
2. organization value outcomes; 
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3. individual performance outcomes; 
4. individual value outcomes. 

Outcomes must be measured for performance and consumer 
appraisal in the area of satisfaction and also for functionality in the 
area of adaptive behaviour in the role status. Evaluations are more 
successful if stakeholders participate in the evaluation and are engaged 
in the decision-making process. Program evaluations are conducted a 
systematic and objective processes that collect, analyse and interpret 
information. More specifically, program evaluation deals with collecting 
and documenting information about a particular program to enable valid 
decision making to a particular aspect of that program (McNamara, 
2000). The ultimate purpose of program evaluation is to arrive at a 
definitive, intelligent, objective and valid conclusion regarding specific 
objectives and questions related to a program’s overall effectiveness 
(Fitzpatrick et al.). 


Effectiveness evaluation 

Effectiveness evaluation strives to answer the question: “Is my program 

meeting its goals and objectives?” It’s primary uses are: 

e compare program’s goals with its achieved outcomes; 

e report the program’s performance and value outcomes; 

e provide formative feedback for program change and improvement 

(Schalock, 2001). 

The difference with program evaluation is that the effectiveness 
evaluation establishes a comparison condition against which accountability 
and outcomes can be judged. 

There are five steps for analysis: 

performance goals; 

purpose and comparison condition; 
methodology; 

data collection and analyses; 

person- and organization-referenced outcomes. 


nA BWN PE 


3. External evaluators, critical friends 


A critical friend can be defined as “a trusted person who asks 
provocative questions, provides data to be examined through another lens, 
and offers critiques of a person’s work as a friend. A critical friend takes 
the time to fully understand the context of the work presented and the 
outcomes that the person or group is working toward. The friend is an 
advocate for the success of that work” (Costa, 1993). 
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The role of the critical friend is therefore a strategic one and can be 
important in assisting improvement. It is essentially a role of support and 
challenge. 

Critical friendship has also been described as less formal than mentoring 
or coaching and probably best described as “a professional relationship 
based on mutual regard and the willingness to question and challenge” 
(Meeting the Challenge: Growing Tomorrow’s School Leaders, 2005). 


4. On the theory of powerful professional development 


Nothing has promised so much and has been so frustratingly wasteful as the 
thousands of workshops and conferences that led to no significant change in 
practice when teachers return to their classroom (Fullan & Stiegelbauer, 1991). 


Reasons why professional development programs appear to be 
ineffective has been subject of research in the United States Staten 
(Darling-Hammond, Wei, Andree, Richardson & Orphanos, 2009). 
Outcome is that every professional development program should be aimed 
at raising student achievement. 

Powerful professional development deepens teachers’ subject-area 
knowledge. Traditionally, a substantial portion of staff development has 
provided teachers with generic instructional skills such as cooperative 
learning separate from specific academic disciplines. Little attention was 
paid to teachers’ knowledge of the subjects they teach and to instructional 
strategies within particular subject areas (Sparks, 2002). 

A number of experts and organizations have suggested that the most 
promising professional development programs or policies are those that: 

e stimulate and support site-based initiatives. Professional development is 
likely to have greater impact on practice if it is closely linked to school 
initiatives to improve practice; 

e are grounded in knowledge about teaching. Good professional 
development should encompass expectations educators hold for 
students, child-development theory, curriculum content and design, 
instructional and assessment strategies for instilling higher-order 
competencies, school culture and shared decision-making; 

e model constructivist teaching. Teachers need opportunities to explore, 
question and debate in order to integrate new ideas into their repertoires 
and their classroom practice; 

e offer intellectual, social and emotional engagement with ideas, 
materials and colleagues. If teachers are to teach for deep 
understanding, they must be intellectually engaged in their disciplines 
and work regularly with others in their field; 
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e demonstrate respect for teachers as professionals and as adult learners. 
Professional development should draw on the expertise of teachers and 
take differing degrees of teacher experience into account; 

e provide for sufficient time and follow-up support for teachers to master 
new content and strategies and to integrate them into their practice 
(Corcoran, 1995). 


4.1. Modelling professional change 


Many programs are based on the idea that changes in belief and attitude 
will cause a change in classroom behaviour. This in its turn would lead to 
improvement in student learning. Gusky (Gusky, Staff development and the 
process of, 1986) provided an alternative model, in stating that significant 
changes in belief and attitudes would only arise after changes in student 
learning outcomes. A teacher tests a change in practice and experience 
a change in student learning. Then they change their belief and attitude 
towards the change (Fig. 1). 


STAFF 
DEVELOPMENT 


Fig. 1 - The teachers development of belief and attitude 


This model can be criticized as being linear. Clarke used the same 
sequence of elements as Gusky but in a cyclic pattern with multiple entry 
points (Clarke D.J., 1988). Challenging teachers approaches, thereby 
creating a cognitive conflict, prior to them attempting to change their 
classroom practice could be a motivator for change (Cobb, Wood & Yackel, 
1990). A further model of the teacher change process was developed by 
Clarke and Peter (1993), and later revised by an international research 
group interested in teacher professional growth as the Interconnected 
Model of Teacher Professional Growth (Teacher Professional Growth 
Consortium, 1994). 
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The Interconnected Model (in Fig. 2) suggests that change occurs 
through the mediating processes of reflection and enactment, in four 
distinct domains which encompass the teacher’s world: 


The Change External 
Environment Domain 


External Source 
of Information 
or Stimulus 
Domain of 
Practice 


Professional 
Experimentation 


—+ Enactment Domain of 
--p Reflection Consequence 


Fig. 2 - The Change Environment 


This model recognizes the complexity of professional growth through 
the identification of multiple growth pathways between the domains. Its 
non-linear nature, and the fact that it recognizes professional growth as 
an inevitable and continuing process of learning, distinguishes this model 
from others identified in the research literature. This model also identifies 
the mediating processes of reflection and enactment as the mechanisms by 
which change in one domain leads to change in another. Any processes of 
professional growth represented in the model occur within the constraints and 
affordances of the enveloping change environment (Hollingsworth, 1999). 


4.2. Organizing professional development 


First content then concept 

The purpose of professional development must guide the way this is 
organized. Effectiveness of one-day-workshops does not appear to be great 
if it goal is to change teacher behavior in classroom situations. First focus 
must be on teacher skills that improve student learning, then a choice must 
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be made for an appropriate way or environment in which this skill can be 
learned. 


Conditions for success 
Five principles for a learning environment for teachers have been 
concluded on a trial-and-error basis. There might be others, not mentioned 
principles. If one of these principles are missing, there might still be a 
successful professionalization strategy. However, experience shows the 
change of success is lower. These principles have only been researched 
in an environment aimed at raising teacher skills in the use of classroom 
formative assessment (Wiliam, Keeping learning on track: Classroom 
assessment and the regulation of learning, 2007): 
e Choice: 
An individual teacher can best determine which development has the 
greatest impact on the learning of his or her student. Therefore the 
teacher must have the choice what to develop. 
e Flexibility 
There are many techniques in formative assessment. These do not have 
to copied but can be changed. Small changes can have great effects. A 
teacher must have the freedom to change fitting the circumstances. A 
teacher who has the opportunity to change a technique, is more inclined 
to use it more often. 
e Small steps 
Teachers learn on the fly, that means during their everyday teaching. 
Therefore changes should be made in small steps. It also gives the 
teacher to practice the technique more often and through repetition 
integrate in a daily practice. 
e Accountability 
There are many factors that influence students achievement. A teacher 
can only influence one, the learning environment in the classroom. 
Research shows that the teacher does make the difference. Therefore 
he or she must be held accountable for the classroom environment. 
The teacher must improve skills to improve students achievement and 
teacher leadership must demand a improvement attitude from teachers. 
e Support 
Consequently to accountability teacher leadership must support the 
teacher in time, focus and facilitate. The school and leadership take 
responsibility in supporting the teacher: 
e create expectation of continuous professional development; 
e keep focus on learning that improves student achievement; 
e facilitate in time, place and support; 
e encourage risk-taking and defend it. 
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4.3. Structure 


An effective way to organize professional development satisfies (most 
of) the conditions mentioned in the previous sections of this chapter. It 
gives the teacher a choice and a structure to improve. It is vital to learn 
with professionals and make room for discussion and sharing ideas. This 
can be done effectively in a Professional learning community. 


Professional learning community 

A professional learning community (PLC) is a method to foster 
collaborative learning among colleagues within a particular work 
environment or field. It is often used in schools as a way to organize teachers 
into working groups of practice-based professional learning. 


If schools are to be significantly more effective, they must break from the 
industrial model upon which they were created and embrace a new model that 
enables them to function as learning organizations. We prefer characterizing 
learning organizations as “professional learning communities” for several vital 
reasons. While the term “organization” suggests a partnership enhanced by 
efficiency, expediency, and mutual interests, “community” places greater emphasis 
on relationships, shared ideals, and a strong culture — all factors that are critical 
to school improvement. The challenge for educators is to create a community of 
commitment — a professional learning community (Dufour & Eaker R.E., 1998). 


Lesson study 

Working in a PLC must be organized as well. Lesson study is a 
professional development process that Japanese teachers engage in to 
systematically examine their practice, with the goal of becoming more 
effective. This examination centers on teachers working collaboratively 
on a small number of “study lessons”. Working on these study lessons 
involves planning, teaching, observing, and critiquing the lessons. To 
provide focus and direction to this work, the teachers select an overarching 
goal and related research question that they want to explore. This research 
question then serves to guide their work on all the study lessons. 

While working on a study lesson, teachers jointly draw up a detailed 
plan for the lesson, which one of the teachers uses to teach the lesson in 
a real classroom (as other group members observe the lesson). The group 
then comes together to discuss their observations of the lesson. Often, the 
group revises the lesson, and another teacher implements it in a second 
classroom, while group members again look on. The group will come 
together again to discuss the observed instruction. Finally, the teachers 
produce a report of what their study lessons have taught them, particularly 
with respect to their research question. 
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Between Stages 4 and 1 
Additional teaching after 

revising the lesson, review video 
documentation, reflect on the 
process after the excitement of the 
public-lesson activities 


Between Stages 3 and 4 
Reflect on actions, make 
possible lesson adjustments, 
develop comments from notes 
made, experience feelings of 
accomplishment 


Between Stages 1 and 2 
Research, brainstorm, go off 

on valuable tangents, explore 
mathematics concepts as teachers 


Between Stages 2 and 3 

Plan pre- and post-lessons in the 
sequence, carefully considering 
student groupings, conducting 
exploratory lessons, acclimatizing 
students to presence of video camera 


Fig. 3 - Lesson Study Cycle 


5. Focus of evaluation 


5.1. Structure of evaluation 


To evaluate the development and effectiveness of the teacher training, 
evaluation guidelines are divided into four categories. The first category 
was aimed at the the training model and how it is derived from the 
questionnaires. This was folowed by an evaluation of the content of the 
training and how this was derived from the questionnaires. The third again 
aimed at the training model from the point of view of the trainer and 
trainee. This was done in two phases. The first phase was in the design 
process, before training was done and secondly after the training. The 
fourt and final category was targeted the content of the training from the 
point of view of the trainer and the trainee again in two phases being the 
design process, before training is done, and after the training. 
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questionairres training model 


- teacher 


Training 
- students 


Fig. 4 - Evaluation guidelines 


5.2. Methodology 


The evaluation as a whole was structured in the before mentioned four 
categories. Detailed design of the evaluation proces and instruments was 
done in four steps. 


Step 1: Defining outcomes 

Outcomes are defined as learning goals of the training: 

I, the trainee, of the training course on formative assessment: 

. know a theoretical framework on formative assessment; 

3. can arrange a classroom situation in which formative assessment is fully 
integrated; 

4. can analyse a classroom situation with predetermined indicators; 

5. can explain why formative assessment is an educational intervention 
with a high rate of effectiveness. 


bre 


Step 2: Evaluation instruments 

For the porpoise of evaluation of the training course researcher created 
and used two questionnaires: 
e questionnaire to be used during the training; 
e questionnaire to be used after the training. 

Since there is a long and a short training designed there will be two 
questionnaires to be used either during a long or during a short training. 
Both evaluation instruments are related to all four levels of evaluation. 


Step 3: Collecting data and drawing conclusions 
From the data collected conclusions will be stated. These statements are 
the basis of step 4, the interview of the trainer and some trainees. 


Step 4: Interview trainers and trainees 
The porpoises of both interviews are: 
e to validate the stated conclusions; 
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e to obtain advice on how to improve the training model and content 
e to get other feedback that could not be concluded from the questionnaires. 


5.3. Elucidation on focus points 


The evaluation of the development and effectiveness of the teacher 
training was done by the use of instruments such as interview and 
questionnaires in combination with an interview. As reasons for evaluation 
we distinguish process evaluation, product evaluation and outcome- 
based evaluation. The latter can be divided in program evaluation and 
effectiveness evaluation 

These instruments have been chosen because they give the most amount 
of information in the most efficient way. 

For each category of evaluation possible evaluators are defined. We 
distinguish external and internal stakeholders. An external stakholder is 
defined as a stakeholder is in no way affiliated with the FAMT&L project 
or any of its participating universities or schools. An internal stakeholder 
internal can be affiliated with a participating university or school, but 
not with the project itself. Table 1 shows the involvement of the different 
stakeholders in the eveluation process. 


Tab. 1 - Stakeholder involvement 


Evaluators/Stakeholder 
Experts on | Teacher | Teachers| Trainees | Trainer 
formative trainers 
assessment 
(external) 
From 
A questionnaires x 
to training 
model 
From 
B questionnaires š % 
to content 
training 
A | Training model x x 
D Content š 5 % è 
training 
E Training x x 
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For each category the goal of evaluation as well as means or 
instruments are given in the evaluation guidelines as well as the reason for 
evaluation. The folowing gives an overview per category. 


From questionnaires to training model 
Goal: to obtain feedback and recommendations on the proposed training 
model focused on objectives and basis on which the model is designed. 
Means: interview. 
Reason: process evaluation; 
product evaluation. 


From questionnaires to content model 

Goal: to obtain feedback and recommendations on theproposed content 
of training model focused on outcomes of questionnaires. 

Means: interview. 

Reason: process evaluation; 
product evaluation. 


C: training model 

Goal: to obtain feedback on the training model from perspective of a 
teacher trainerand a teacher. 

Means: questionnaire; 
interview. 

Reason: process evaluation; 
product evaluation. 


D: content training 
Goal: to obtain feedback and recommendations on the proposed content 
of training model. 
Means: questionnaire; 
interview. 
Reason: product evaluation; 
outcome-based evaluation; 
program evaluation. 


E: training 
Goal: To obtain feedback and recommendations on the training 
Means: questionnaires; 
interviews. 
Reason: product evaluation; 
outcome-based evaluation; 
program evaluation; 
effectiveness evaluation. 
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6. Working with critical friends 


In the cycle of development of the FAMT&L-project critical friends are 
involved at the final stage of design. The role of the critical friends is to 
question and challenge from the perspective of a teacher-trainer as well as 
teacher in order to support the reflective process and inform the mid cycle 
self-assessment. 

At this stage of the project the critical friends provide challenges on the: 
e design of the training; 

e content of the training. 

Therefore their expertise lies in: 

e teacher trainer with expertise on curriculum design; 
e in service teacher. 

The following model is intended to structure the cooperation between 
the consortium and the critical friends. It offers a structure within which 
issues can be clarified and actions can be formulated. 


Step I: 


Step 2: 


Step 3: 


Step 4: 
Step 5: 


Step 6: 


Step 7: 
Step 8: 


Critical friends are informed about the: 

e approach of the consortium; 

e outcomes to date; 

e design of training; 

e content of training in relation to outcomes of questionnaires. 
Critical friends are informed about their proposed role and scope 
of challenges asked by the consortium. 


Critical friends question and challenge the consortium in a face- 
to-face-meeting. Together, the consortium and critical friends 
analyse the risks, alternatives and consequences of each potential 
solution. 


Consortium nominates solutions which should be progressed. 


Together, the consortium and critical friends list all possible 
obstacles to achieving the solution. 


Critical friends assists to brainstorm possible strategies to 
minimise/overcome these obstacles. 


The consortium devise an implementation plan. 


Together, the consortium and critical friends review the outcomes 
of the action plan and continue the process until all relevant 
issues are worked through/resolved. 
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6.1. Scope of challenges 


In this paragraph the scope of the challenges by both critical friends are 
directed and questions to guide the critical friend are given. 


Product evaluation 

Question: Does the approach of the consortium guarantee a: 

e effective; 

e sustainable; 

e transportable; 

training both for in-service teachers as well as students training to become 
a teacher? 


Outcome-based evaluation 

1. Program evaluation 
Question: Does the training model guarantee the specified and desired 
outcomes of the training? 

2. Effectiveness evaluation 
Question: Does the proposed content of the training guarantee reaching 
the specified goals and objectives? 

3. Impact evaluation 
Question: Is the proposed training of added value for in-service teachers 
as well as students? 


Program evaluation 

1. Organization performance outcomes. 

2. Individual performance outcomes. 
Question: Is the proposed training functional for the purpose it was 
made, being the possibility to train participants with the use of a 
database of examples and the possibility of distance-learning? 


6.2. General advice from critical friends 


Professional development is essential for teachers to progress as teaching 
practitioners. But there are many challenges in integrating professional 
development into regular practice. How to get the most from the training? 


Content of the training 

Teach what you preach. Consider the importance of teaching in a 
formative way in the pilot course. Put more elements of FA into the course. 
Walk the talk. What is the heart of assessment for learning? 


153 


Copyright © 2018 by FrancoAngeli s.r.l., Milano, Italy. ISBN 9788891774637 


e Self-regulation: One of the skills teachers and students should develop 
is reflection on learning ore self-assessment. Trying to take in new ideas 
and embed them into your practice is not easy. What helps is reflection. 
Focus on successes, beware of pitfalls. 

e Self-regulation and self-directed: 


Because adult learners are self-directed and pragmatic learners, their primary 
concern is the anticipation of and testing the usefulness of what they learn to 
improve practice rather than preoccupation with the background theory and 
general information. This makes them selective, yet, focused on what and 
how they would like to learn, thus, exercising more control over their learning 
(Elliot & Campbell, 2013). 


e The face to face time is very valuable. Use it effectively by “flipping 
the classroom”. This means for example that the participants watch the 
video or read the theory at home. So there is more time to apply the 
knowledge in the face to face meetings with others. And there is more 
time for collaborative learning and interactive dialogue. 

e Let the participants practice. They will learn FA by doing, by 
constantly having new experiences and attempting to integrate those 
experiences. 


Design of the training 

Make the teaching (and the model of the training) and learning activities 
as well as the formative assessments task aligned to the outcomes, in order 
that the participants are helped to achieve those outcomes more effectively. 
This is called designing constructively aligned outcome-based teaching and 
learning (Biggs & Tang, 2007). 

Assessment, also formative assessment is criteria referenced ore based. 
How well a participant (teacher) performance compares to the criteria in 
the outcome statement. Pay attention to describe the learning goals in a 
way that tells us how we would recognize if or how well participants have 
learned what is intended they should learn or be able to do so. Make it as 
explicit as you can and share the learning goals with the participants. 


6.3. Conclusions from sessions with critical friends 


It was noted there are many differences between the five designed 
training courses. The consortium first worked along the line that there 
should be a common training in every country. The end-result should be a 
validated training course, based on the principles of formative assessment, 
cooperative learning with peers. 
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Each partner took their individual circumstances into account and 
designed the training course. All courses were based on the design made 
by the Swiss partner. Therefore each partner did do just to the basic 
principles as formulated at the beginning of the project. 

After the session with the critical friends the consortium in cooperation 
with the critical friends formulated a common core for each training. 


6.4. Micro-simulation 


Introduced by the partners from Cyprus and Switzerland, in each 
training there will be a practical component called Micro-simulation. 
Micro-simulation was defined: 


Simulating a lesson practice before a group of peers, where a designed 
implementation is used with the purpose of experimentation, feedback and 
improvement. 


It is important to have a practical component in the training. Ultimate 
goal is to change the professional behaviour and the skill of implementing 
formative assessment should be practiced. Doing this with micro- 
simulation this also has the advantage of collaborative learning, peer- and 
self-assessment. 


7. Use of formative assessment 


Walk the talk. Since the outcome of the training should be a more 
competent teacher in the use of formative assessment, the training itself 
should be based on the same principles. This also contributes to the 
element of a Life Long Learning (Fig. 5), a teacher able to define one’s one 
learning-need and the ability to assess the progress one’s own learning and 
a way to improve one’s own learning. 


Use of questionnaire 

An almost direct result of the choice to use formative assessment in 
the training is the questionnaire at the beginning of and at the end of the 
training. In this way the trainer builds an image of the present knowledge 
from where to start the training. The trainee sets one’s own learning goals 
and defines success criteria. These are then made visible at the end. The 
starting point in each country is very different. It was therefore concluded 
there should not be a completely common questionnaire. 
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What do we 
want the learners 


4 have learnt? 


Xt, 


4 
MSsassY 


Fig. 5 - Life Long Learning elements 


Use of video 

The learning through video is a new and uncharted territory. First 
results in research show this an effective way in teaching skills such as 
professional teacher behaviour. This was also one of the basic principles on 
which the project is based. In every training there is made significant use 
of videos showing practices of formative assessment. These videos will be 
analysed by the grid as defined during the project. The same grid is used 
to analyse the design of formative assessment by the trainee. 


71. Dilemma’s 


Privacy concerns using videos 

There remains the problem of privacy and consequent problems with 
the use of self-made videos and placing them in the web repository. This 
in some participating countries creates a bigger problem than others. 
Especially in the Netherlands the situation is that there are no videos 
available made during the project for use in the training. This was 
discussed with the critical friends. The solution defined is the use of videos 
available on the internet, mostly from English speaking countries. This 
resolves the problem of the lack of videos. Problem that is not solved is 
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the building of the web repository. Unfortunately, the sensitivity of using 
videos in the public domain in the Netherlands, could not be solved during 
this project. The devised solution was shared within the consortium and 
with the critical friends and seen as appropriate. 


Teacher resistance 

Teachers almost all have a traditional, conservative reaction towards 
everything considered new. It is difficult to change (professional) 
behaviour, let alone the behaviour of the teacher. Traditionally, teacher 
training is done from a push-strategy. A universal message is at the 
centre of the training. These kinds of trainings show almost now lasting, 
sustainable effect. A pull strategy, where the trainee is placed at the centre 
of one’s own learning, and is aimed at the individual need of the trainee, 
leads to less resistance and a more positive effect. 


7.2. Evaluation outcomes 


To generate conclusions from five individual evaluations and two 
external evaluations, all items from each evaluation were correlated with 
others. Items that occurred in more evaluations then were named as overall 
conclusions. Consequently, there were items seen in evaluations that were 
specific to one evaluation. These can be named, and in fact important, as 
strong of weak points or recommendations for that individual training, not 
for the overall training course. 


8. Conclusions 


What can be concluded from all training evaluations is that the 
designed pilot training course was positively received by participants and 
named as: 

e well organised; 
e clear goals. 

Participants showed growth in their theoretical knowledge on formative 
assessment, its purpose and application. Their belief was influenced in 
a significant way. It cannot be concluded that as a consequence teacher 
behaviour has changed sustainable. Teachers feel the need for more 
examples, experimentation and practical learning. 

Learning in small groups with a lot of interaction, sharing ideas and 
discussions had a very positive effect on participants belief, motivation and 
commitment. 
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Tab. 2 - Product - Outcomes - Program 


Evaluation 


Product evaluation 


Does the approach of the consortium 
guarantee a: 

e effective; 

e sustainable; 

e transportable; 

training both for in-service teachers as well 
as students training to become a teacher? 


It can be concluded the training is effective 
in changing belief and attitude of teachers 
on the subject of formative assessment. In 
its current design and examples it is aimed 
at teachers in mathematics, but the model in 
itself can easily be transferred to be used for 
teachers in every other subject. The training 
is sustainable as it has a formative character 
and foremost a practical component. 


Teacher behaviour could be changed more 

sustainable through: 

e more time allotted to experimentation and 
observation; 

e more time between meetings; 

e training over a longer period. 


Outcome based evaluation 


Program evaluation: 

Does the training model guarantee the 
specified and desired outcomes of the 
training? 


The desired outcome of the training was 
observed in all training courses, named by 
trainers, trainees and external observers. 


Effectiveness evaluation 

Does the proposed content of the training 
guarantee reaching the specified goals and 
objectives? 


In most the training objectives were 
reached. Improved can be the ability 
of participants to observe and reflect on 
examples of formative assessment in the 
classroom setting. This can be done by 
more experimentation and practice and 
would require more time. 


Impact evaluation 
IS the proposed training of added value for 
in-service teachers as well as students? 


All participants unanimously stated the 
training is a major improvement on their 
everyday practice. The feel they learned 
a lot and are motivated and committed to 
learn more. 


Program e 


valuation 


Is the proposed training functional for the 
purpose it was made, being the possibility 
to train participants with the use of a 
database of examples and the possibility of 
distance-learning? 


As the positive effect of learning in a 
learning community is mentioned both by 
the participants as the trainer, it can be 
concluded that especially this part of the 
training is seen as a contributing factor 
to its success. This could imply that long 
distance learning would not be effective. 
The database and examples can be used 
in learning in a learning community in an 
effective way. Also mentioned is the need 
for an expert to structure the learning of the 
participants in the learning community. The 
grid for analysing videos and practice is too 
large and complicated. 
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8.1. Strong points of the training course 


There are a lot of detailed items that can be named as strong points in 
the training. These items can be collected in four categories: 


Use of videos 

The use of videos as an indispensable part of the training. They 
give examples of formative techniques directly applicable in the daily 
practice of the participant. It gives participants the opportunity to see 
formative techniques being used in that daily practice and reflect on it 
using the devised grid in the project. This way participants understand 
the use an can make the transfer to their own teacher behaviour in small 
successful steps. They are able to control their own learning which creates 
commitment and motivation. The discussion based on the short teaching 
episodes displayed was fruitful and served as representative examples of 
applying formative assessment in the classroom. 


Learning community 

The design of the training is all about interaction. Participants learn 
a great deal in discussing, designing and reflecting. This makes the 
training in the most part formative. Learning in a community gives 
the opportunity to exchange opinions and ideas about teaching and 
assessment. 


Practical component 

The domain of practice is present in the training: 
e microsimulation; 
e in practice in classroom. 

This gives the opportunity to apply formative assessment in the 
classroom, improve on and experiment. It fills the need for teachers to learn 
by practice. 


Change through practice 

Based on our observations and the participants’ comments and 
feedback, after the training program the pre-service and in-service 
teachers’ beliefs moved positively towards the purpose of formative 
assessment. Furthermore, their beliefs about their skills in applying 
formative assessment through a diversity of techniques turned also 
towards a more positive direction, as the participants expressed to have 
stronger skills. A change in focus between practice and theory, strengthens 
knowledge skill and participation. 
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8.2. Weak points of the training course 


The weakest point in the training is time. From literature on learning 
and learning or teaching practice we know this takes time. Time must be 
seen in different contexts: 


Time for training-sessions 

An effective amount of time for a sessions seems to be 1,5-2 hours. It 
is better to have more meetings on different dates than to create a long 
meeting. 


Time for practice 

It takes time to experiment and improve. After this it takes time to 
integrate a teaching behaviour and automate it in one’s arsenal of teaching 
strategies. It needs practice to learn the practice of a formative strategy and 
that takes time. 


Time for examples and experimentation 

There is a need for examples and experimentation. Within this 
experimentation there is a need for time to evaluate end reflect. This way 
of working needs time to learn. 


Time between training-sessions 

An integral part of the training is to practice in the classroom, observe 
and reflect. This takes time. Teachers have to have time between meetings to 
do this. In literature a period of 5-6 weeks between meetings is mentioned. 


8.3. Recommendations 


Time 

As described in paragraph 6.5.3 the training can get more effective by 

time-management: 

e time for training-sessions; 

e time for practice; 

e time for examples and experimentation; 
e time between training sessions. 

It is recommended to design sessions of 1,5-2 hours with intervals of 
5-6 weeks. In between these sessions teachers must experiment in and 
reflect on their practice. These experiments should be part of the sessions. 
This learning can be done very effectively in a small teacher learning 
community. 
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Repository 

Strengthen the use of the repository by enlarging the number of 
videos. Describing how to use the videos with small descriptive tasks 
can make the use of the repository effective. Training trainers to use the 
repository can enlarge the scale of use quickly and efficient. One can train 
participants in the pilot training course to use the repository in schools 
(becoming experts as trainer for future training-sessions). 


Simplify grid of analyses 

The grid is found to large and complex. There are too many items that 
are connected to formative assessment but can also be described as normal 
teacher behaviour. As a consequence the effectiveness of reflecting with 
a large grid is not as high as it can be. Focus of the grid should be on 
aspects of formative assessment, such as questioning, feedback. Option 
could be to define grids specific to strategies and techniques of formative 
assessment. 
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10. Exploitation activities related 
to the FAMT&L project 


by Alessandro Gimigliano* 


1. Introduction 


The exploitation activity for the project FAMT&L — Formative 
Assessment in Mathematics for Teaching and Learning (538971-LLP-1- 
2013-1-IT-COMENIUS-CMP) funded by the Lifelong Learning Program 
of the European Commission started with the preparation of a work plan, 
modified “in progress” during the time of the project. It presented the 
exploitation plan for the 3-years FAMT&L project and it has been a 
reference document, which described the strategy and the planned tools 
for the exploitation of the project results. It also showed the different 
directions of the exploitation and gave an overview of the expected project 
results and the key stakeholders in the project. In order to achieve the best 
possible results different hypothesis of exploitation have been analyzed. 
The exploitation plan took all project partners into account and it was also 
linked to their outcomes. 

The ambition of FAMT&L has been the development of a training model 
based on the use of videos and tailored thanks to the results of surveys 
among both teachers and students. This preliminary work and the analysis 
done has brought to the organization of pilot courses and to the making 
of a web repository for the video materials. The priority of the project was 
to promote the training of teachers in order to develop the basic skills of 
students in mathematics, a field of scientific education which is particularly 
important to ensure the right to citizenship for the new generations. 

Teacher training, in FAMT&L, focuses on a crucial element of 
innovation for the effectiveness of school systems: conceptions and 
practices of (math) teachers about learning assessment of students and the 
possibilities given by using formative assessment (FA). 


* Alma Mater Studiorum Universita di Bologna. 
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The exploitation focuses on development and enhancement of 
teacher’s competence. The special aim of the exploitation, involving all 
stakeholders, is to develop ideas able to reduce the gap between the ideals 
of a democratic school in the involved Countries and the present level of 
the use of assessment practices (our ideal is to have a school system which 
is able to use FA to have most of the pupils reaching “good levels” of 
knowledge, apt to help them to be responsible citizens). To apply and use 
the results of the FAMT&L project in practice — both during the time of 
the project and in the future — should enhance the ability of the teachers to 
fight such gap. 

Any project can be successful only if all participating parties 
are working in close cooperation; our plan, part of Work package 8 — 
Exploitation has been collecting the outcomes of the project and advise 
how these could be exploited. In this chapter we describe how all the 
exploitation strategy developed. 


2. Objectives and exploitation strategy 


To achieve the above-described goals, the exploitation strategy was 
to identify important organizations, fairs, meetings, workshops and 
events to which the FAMT&L project shall contribute and participate 
(in addition to other tools for dissemination of information and results 
better defined in the dissemination strategy). The plan has been to make 
use of our dissemination activities as a complement to reach relevant 
stakeholders and stimulate dialogue on the topics of the project. The 
plan defined the modality for the realization of briefings with policy 
and decision-makers in the educational field at national level in order 
to create synergies that will guarantee a successful exploitation of the 
project’s results. 

The exploitation strategy planned to: 

e continuously interact with new and already involved stakeholders in 
order to define an on-going strategy for the exploitation of results; 

e define a tailored exploitation plan that will make use of dissemination 
activities as a complement to reach relevant stakeholders, stimulate 
dialogue on the topics addressed in the project and modify the 
education policy in the direction of efficacy and effectiveness; 

e implement the sustainability plan encouraging continuity after the co- 
financing by the EU will end. 

The exploitation plan was made of several objectives and required a 
methodology. 
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The exploitation objectives were: 

e To develop a network of researchers and teachers that, working 
together, would improve, both theoretically and practically, their 
knowledge and skills about FA. 

e To implement and to keep active an e-learning platform that could be 
used by both teachers and researchers as a video educational document 
repository, usable and sharable with all colleagues, also in the future. 

e To find partners (teachers, researchers and other target groups) who 
would apply and use the results of the FAMT&L project in their practice. 

e To spread the training model of FA in mathematics (in synergy with 
the use of the platform) to groups of school teachers of the different 
partner countries (and others). 

e To ensure that all the knowledge created and all the practices of 
formative assessments elaborated within the FAMT&L project would also 
be used in future. 

To bring different strategic actions closer together has increased the 
likelihood of a solid impact for the FAMT&L project outcomes and results. 
In order to generate the required impact and visibility all activities had to 
be well coordinated: Thus the exploitation plan ran parallel to the phases 
of the work program and adjusted itself to the main activity of each phase. 


3. Methodology 
3.1. The essential steps 


e Elaboration of an Agreement of Cooperation with schools 

In order to strengthen the relation and collaboration between 
Universities and Schools for the application of the teachers’ training model, 
an Agreement of Cooperation has been realized and proposed to the 
schools involved with the project. The agreement set up the basis for the 
common work on the project. 


e Collecting and studying data via questionnaires 

To have a significant view of the awareness about FA among teachers 
and students, we prepared and distributed questionnaires to teachers and 
students of the cooperating schools. Their analysis confirmed the need of 
better actions to improve the use of FA in the teaching of Mathematics. 


e Short videos and Pilot Courses 

Short videos for the exploitation of projects results have been realized, 
in particular about experiences analyzed in WP2 and WP3 and the 
activities carried out in WP4. 
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This is the key action of our project: promoting teaching and learning 
about FA via video analysis. The training model has been implemented in 
Pilot Courses organized by all the nodes of our projects. Moreover short 
videos and other materials that are stored in our web repository could be 
used also in the future and can already be used in the process of training 
future teachers (as university students in mathematics or education). 


e Quality check 
“Critical fiends” have been present in several phases of the work, in 
order to monitor the project and collaborate to its Quality Plan. 


e Briefings with policy and decision-makers in the educational field at 
the national level 
Our consortium has been creating synergies in order to guarantee a 
successful exploitation of the project’s results through briefings with policy 
and decision-makers and educational agencies at all national levels and 
integrating FAMT&L’s and other institutions’ initiatives. 


e The Final Conference 

The Final Conference of the project, in Bologna, has gathered national 
agencies, schools, teachers and graduate students; the aim was to have it not 
only as a mean to illustrate the previous work done, but also to promote a 
knowledge network with participants from many sectors (teachers, trainers, 
students, professionals, etc.). 


3.2. The two dimensions of our activities 


The exploitation’s events and activities have been based on two dimensions: 
1. Horizontal dimension 
Within the FAMT&L network the state-of-the art of the work has been 
continuously discussed among all project partners (to define priorities, 
impact criteria and use of the projects achievements in accordance with 
the project’s objectives). The Exploitation Plan and the dissemination 
activities have been tools to reach relevant stakeholders and stimulate 
dialogue on the topics addressed in the project. 
2. Vertical dimension 
There are scholars at Universities in different countries teaching and 
researching about the use of FA in teaching; they have been contacted 
and invited to a dialogue to ensure continuity in the knowledge of the 
state of the art for the whole network; in this way new methods and 
results can be easily exchanged to build up a basin of shared scientific 
knowledge from which to draw upon information and experiences. 
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4. Highlights of project’s results 


The main issue of the project was to promote dissemination of good 


practices of formative assessment in the teaching of mathematics in low 
secondary school. 


The promotion of a new method of training for teachers is a concrete 


result which has been pursued mainly through FAMT&L website 
and network, but also with interviews and articles, seminars and with 
participation at conferences. 


4.1. Publications 


In particular, the work of our researchers has produced several 


publications and presentations (see www.famt-l.eu/publications/): 


Paraskevi Michael-Chrysanthou, Athanasios Gagatsis & Ira Vannini. 
Formative assessment in mathematics: A theoretical model. Acta 
Didactica Universitatis Comenianae Mathematics, issue 14, 2014. 
Paraskevi Michael-Chrysanthou. H diapopga@tixy adciokdynon otn 
didaoradia Kal ua0non tæv ua0nuatixov (Formative assessment in 
Mathematics for teaching and learning) XII Pedagôgiai Értékelési 
Konferencia, 2014. 

Laurent Jeannin, Iuliana Rossi, Athanasios Gagatsis, Paraskevi Michael, 
Ira Vannini, Giorgio Bolondi, Federica Ferretti, Laura Tartufoli, Silvia 
Sbaragli, Miriam Salvisberg, Rob Velder. Quelle évaluation formative 
en mathématique au sein de ces 5 pays: Italie, Chypre, Suisse, 
Hollande et France: Projet Européen FAMT&L. ADMEE, 2015 (PPT 
presentation). 

Federica Ferretti, Stefania Lovece. La valutazione formativa per la 
didattica della matematica nell’ambito del progetto FAMT&L. Le 
concezioni degli studenti di scuola media nei confronti degli strumenti 
di verifica utilizzati in classe. In Ricerche di Pedagogia e didattica, 
vol. 10/2015, no. 2, pp. 39-68. 

Paraskevi Michael-Chrysanthou, Stefania Lovece & Ira 
Vannini. Exploring teachers’ beliefs on formative assessment 
in mathematics teaching and learning in Cyprus and Italy 
Conference — EAPRL, 2015. 

Stefania Lovece, Ira Vannini & Paraskevi Michael-Chrysanthou. 
Methodologies and tools for the video analysis of formative assessment 
practices in classroom (with students aged from 11 to 16) Conference — 
EAPRL, 2015. 
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e Giorgio Bolondi, Federica Ferretti, Stefania Lovece and Ira Vannini 
(Alma Mater Studiorum Universita di Bologna), Elena Franchini, Miriam 
Salvisberg and Silvia Sbaragli (DFA-SUPSI of Locarno). The formative 
assessment in mathematics education. First results of an international 
project. National Conference Encounters with Mathematics n. 29, Castel 
San Pietro Terme, Bologna (6-7-8 November 2015). 

e Bolondi, G., Ferretti, F., Gimigliano, A., Lovece, S., Vannini, I. The 
Use of Videos in the Training of Math Teachers: Formative Assessment 
in Math Teaching and Learning, in: Integrating video into pre-service 
and in-service teacher training, Hershey PA, IGI Global, in corso di 
stampa, pp. 128-145. 

e Vannini, I., La Valutazione Formativa in classe: formare la 
professionalità degli insegnanti di matematica attraverso la 
videoanalisi. Un percorso di ricerca all’interno di un progetto 
internazionale. Presentation at International conference: “Video digitali 
e formazione egli insegnanti”. Cagliari (Italy) 1 luglio 2016. 

e Lovece, S. The use of video in a teacher training course to promote 
the correct use of formative assessment for improving Mathematics 
teaching and learning, in Educating the Best Teachers: a Challenge for 
Teacher Education Proceedings of the 41st Annual ATEE Conference, 
Brussels, Belgium, ATEE (Association for Teacher Education in 
Europe) 2106, pp. 106-114. 

e Gagatsis, A., Michael-Chrysanthou, P., Christodoulou, T. & Elia, 
I. (2016). Formative Assessment in The Teaching And Learning of 
Mathematics in Cyprus. Nicosia: University of Cyprus. 

e Dozio, E., Franchini, E., Salvisberg, M. & Sbaragli, S. (2015). Le 
convinzioni di docenti e studenti sulla valutazione formativa in 
matematica. Scuola ticinese, 3, 27-30. 

e Laurent, J., Salvisberg, M., Vannini, I., Sbaragli, S. Méthodologies et 
outils pour l’analyse vidéo de pratiques d’évaluation formative en 
classe. Un projet de recherche internationale, ADMEE, 2016, Lisbone. 

e Michael-Chrysanthou, P., Christodoulou, T., Elia, I & Gagatsis, A. 
(2016). Multiple semiotic means in the use of formative assessment in 
secondary school mathematics. La matematica e la sua didattica, 24(1- 
2), 125-144. 


4.2. Participations in conferences and meetings 


Researcher from our network participated to the following meetings: 
e XII Pedagógiai Ertékelési Konferencia — 2014 Paraskevi Michael — 
Chrysanthou. H Gauopoonkxn atioA6ynon oty didackadia koi uaOnon 
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tov Ladnuatixe@yv (Formative assessment in Mathematics for teaching 
and learning). 

EAPRIL 2014 (Paraskevi Michael — Chrysanthou, Athanasios Gagatsis 
Students’ beliefs for formative assessment in Mathematics teaching and 
Learning. 

ADMEE, 2015 (Laurent Jeannin, Juliana Rossi, Athanasios Gagatsis, 
Paraskevi Michael, Ira Vannini, Giorgio Bolondi, Federica Ferretti, 
Laura Tartufoli, Silvia Sbaragli, Miriam Salvisberg, Rob Velder). 
Quelle évaluation formative en mathématique au sein de ces 5 pays: 
Italie, Chypre, Suisse, Hollande et France: Projet Européen FAMT&L. 
National Conference “Encounters with Mathematics n. 29”. Castel San 
Pietro Terme, Bologna (6-7-8 November 2015). 


The following meetings were essential steps of our project: 

Project meeting in France (September 2014). Discussions about how 
to conduct the analysis of videos. Decisions about the grid for analysis, 
web platform. 

Project meeting in Holland (February 2015). Definition of the grid for 
the video analysis and study of data from questionnaires. 

Project meeting in Cyprus (September 2015). Conclusion of analysis of 
data from questionnaires, summary of the results and discussion about 
the use of them for designing our training courses. Presentation of next 
phases of the Project and outstanding issue. 

Project meeting in Switzerland (April 2016). With the presence of 
academic and scientific experts as critical friends and for the evaluation 
of the state of the art of activities and preparation of the Europe Final 
International conference. 

Final Conference in Italy. It was the occasion to present the final 
results, create new opportunities for exchanging ideas and experiences 
and decide about the future of what has been realized. 

Notice that every event (conference, seminars, meetings, etc.) organized 


within the project has been reported on our site (www.famt-l.eu). 


4.3. A competition 


The FAMT&L group organized a contest aimed to enhance the 


creativity of math teachers by encouraging them to use mathematics in 
an innovative way of expression and communication through Formative 
Assessment. The particular format of the contest is that the participants 
(teachers) are to elaborate a short video with their students creating small 
situations of Formative Assessment in math classrooms, so to their interest 
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towards the implementation of Formative Assessment techniques and 
practices. 
Here is a brief description of how the competition was organized. 


The Product 
The product had to be a short video (max 5 minutes) of a teacher (and 

students), presenting one of the following situations: 

e providing a definition/a description of Formative Assessment with any 
means (i.e verbally, using pictures, diagrams or other representations, 
with movements etc.); 

e proposing a specific Formative Assessment technique to be used in 
mathematics teaching; 

e presenting a short episode of the implementing Formative Assessment 
in classroom, in which a particular technique is used. 


Participants 

e The participants could be secondary education math teachers from 
public and private schools. 

e Each participant was to represent his/her school. 


Steps of the contest 

1. Each video had to be uploaded on the competition website. 

2. The best 2 participants from each country have been selected, without 
ranking, by each partner country. 

3. All the selected participants’ videos from each country have been 
judged in the final stage from the Jury. 

4. At the conference in Bologna, the Jury announced the First, Second and 
Third Prize. 


The Evaluation 

The selection of the winner in the competition and the other finalists 
was conducted in the final meeting a day before the final conference. In 
this phase of the meeting three people of team of Swiss, three people 
of the Cypriot team, four of the Italian team and two researchers from 
Netherlands were present. French team didn’t participate in this procedure. 
Concerning the procedure of the evaluation to choose the finalists, four 
criteria were set: 
1. relation to formative assessment; 
2. the role of the teacher; 
3. effectiveness of the interaction between teacher-students; 
4. effectiveness of the interaction between students. 

For each criterion the members of the Jury could assign a mark ranging 
from 1 (low grade) to 3 (high grade) points. 
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The Jury 

Giorgio Bolondi, Theodora Costantinou, Seerp De Blauw, Federica 
Ferretti, Elena Franchini, Athanasios Gagatsis, Alessandro Gimigliano, 
Fatiha Lograda, Stefania Lovece, Miriam Salviesberg, Silvia Sbaragli, Ira 
Vannini, Rob Velder. 


Prizes 

e First Prize: 500€ and certificate of success. 

e Second Prize: 300€ and certificate of success. 

e Third Prize: 200€ and certificate of success. 

e The prizes have been given as equipment and material to be used in the 
winners’ schools (such as computers, tablets, interactive white boards 
e.t.c). 


The winners 

First prize. Netherlands: Luzac lyceum Amsterdam, Uiterwaardenstraat 
263, 1079 CR Amsterdam. 

Second prize. Switzerland: Scuola media di Gravesano, Via Str. Regina 
6929, Gravesano, Canton Ticino, Svizzera. 

Third prize (ex aequo). Cyprus: Olympion Private School Cyprus, 25 
Martiou 87, Palaiometocho 2682. 

Third prize (ex aequo): Italy: Istituto Comprensivo N.5 Bologna, via A. 
di Vincenzo 55, 40129, Bologna. 


4.4. Activities towards key stakeholders 


The SUPSI (Switzerland) activities 

Collaboration with DECS (Dipartimento educazione cultura e sport), 
which has also co-funded some FAMT&L activities). 

Meetings with Experts in mathematics for middle schools from DECS 
to explain the project. 

Meetings with responsible and colleagues for training in DFA /SUPSI 
to explain the project and find solutions to make a pilot training course for 
future teachers. 


The UCY (Cyprus) activities 
During the project’s realization the group has contacted several national 

agencies. 

e The University of Cyprus (UCY), where students and professors have 
been informed about the actions (i.e. organization of a seminar about 
formative assessment and the FAMT&L project) of the project. In the 
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UCY there are different faculties that deal with formative assessment 
and cooperation with them has promoted more the FAMT&L project. 
Cyprus Mathematical Society (CMS) is another agency which promoted 
the FAMT&L competition and the training courses. Through the CMS 
a 2 hours Symposium has been organized about the aim and the actions 
of the project and it was held within the actions of the 18" Cyprus 
Conference on Mathematics Education and Science. The purpose of this 
Symposium was to promote our training model and invite teachers to 
participate. 

Olympion Private School Cyprus is another agency, where the actions 
of the project were presented and the training model was promoted. 
Cyprus Pedagogical Association (CPA) is probably the oldest 
Pancyprian pedagogical research association. As part of its activities 
CPA organizes a biannual conference in order to provide opportunities 
to researchers, students and educators in Cyprus to present their 
educational research work. With this organization the FAMT&L 
actions and some of the questionnaires’ results has been published in a 
conference and the collaboration will continue by publications with this 
organization and in its website. 

The Cooperation with the Ministry of Education (inspectors of 
Mathematics) has also given to the group access in schools where the 
FAMT&L project, its objectives and its results has been promoted, 
focusing on the training model (and this activity is still going on). 
Cooperation with Greek Universities, mainly with the National and 
Kapodistrian University of Athens. The group has organized and still 
will organize seminars in Greece, in which the actions of the project 
and its results can be presented. 

The Hellenic Mathematical Society (HMS) is a learned society which 
promotes the study of mathematics in Greece. This agency organizes 
an annual conference in order to provide opportunities to researchers, 
students and educators to present their educational research work. With 
this organization we have published materials about the FAMT&L 
actions and some of the questionnaires’ results and we aim to continue 
the publications with this organization and in its website. 

Cooperation with Greek Universities, especially with the 
‘National and Kapodistrian University of Athens’, the ‘University 
of Rhode Island’ (Department of Education and Mathematics- 
professor Avgerinos Evgenios) and the ‘University of Macedonia’ 
(Thessaloniki-teacher Anastasiadou Sofia). We will continue to 
organize seminars in Greece, in which the actions of the project and 
its results will be presented. 
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The UNIBO (Italy) activities 
In the years of the project we had many contacts with several kinds of 

possible stake-holders: schools, researchers, institutions, publishing houses. 

The interest about the outcomes of our project has been high, and we have 

worked and still are working to seek collaborations with regard to two 

objectives: 

1. to keep the web repository active, to enlarge its contents (videos and 
video analysis) and to give it a new institutional position; 

2. to continue the research about the effectiveness of our model 
of formation via video-analysis (to check its influence in changing 
evaluation practices and ideas). 

As for point 1), Unibo is working to make the sustainability of the 
repository possible and is exploring the ways for collaboration with several 
agents: 

e UNIBO itself (find funds and personnel to manage the web repository); 

e INDIRE (National Institute for Documentation, Innovation and 
Educational Research), the first research agency of the Italian Cabinet 
of Education which has shown interest to be involved in keeping and 
using the web repository; 

e Publishing houses which have a sector dedicated to teachers’ formation. 
Such actors could be involved in the publication of manuals for video 
analysis based on the materials of our web repository and (maybe) keep 
samples of it in their websites. 

Our aim is to guarantee that in the future our web repository will be 
hosted on a platform which is adequate for teachers formation and such 
that its use for training activities will kept and developed. 

As for point 2), contacts, exchanges and agreements are ongoing with 
several actors to keep a research network active and developing. 


The UCP (France) activities 

1. Collaboration with the Department of Education, especially for the 
organization of formal and informal meeting with researchers and 
teachers. Cooperation also with some professors and researchers 
of Mathematics Education, which provides a scientific supervision 
of the use of tool and techniques of formative assessment in 
mathematics. 

2. Contact and exchanges with several schools and, in particular, with 
mathematics teachers to explain the project and to delineate teachers’ 
formative needs. 

3. Cooperation with the Ministry of Education (inspectors of Mathematics) 
to collaborate for FAMT&L Project achievement objectives and its 
exploitation. 
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The InHolland (Netherland) activities 


The following table describes the contacts and activities of the 


InHolland group. 


Tab. 1 - InHolland activities 


Name of agencies 
Beta-partners 


Description 


Network meetings with other universities and High 
Schools with purpose: 

e exchange knowledge; 

e in-service training; 

e think-tank. 


SLO, VO-raad, Plexs 


SLO: supporting organization for schools for 
development of curriculum design. 

VO-raad: council for high school boards. 

PLEXS: council for examination. 

Starting as a think-tank (initiated by ministry of 
education) to research current use of formative 
assessment and how to organize professionalization 
of school boards, examination-administration and 
teachers. 

So, actually three agencies, working both together 
and also apart on this subject. 


Schoolinfo 


Supports schools in implementing strategies for 
differentiated and personalized learning in a network 
of various partners. 

Organizes conferences, seminars and develops in- 
service training in professional learning communities. 


Ministerie van Onderwijs 


Ministry of Education, working with the project group 
on formative assessment. 


Samenwerking 
Amsterdamse 
Lerarenopleidingen SAL) 


Cooperation between the Institutes for Teacher 
Education (SAL): University of Amsterdam, Free 
University, Netherlands’ Hogeschool InHolland of 
Applied sciences, Amsterdam University of Applied 
sciences, IPABO Teacher Training College. 


ADEF Assessment 


General Consultation between Directories of 
Educational Faculties. 


4.5. Development of strategies 


To keep the project results active and exploit its success by all 
participants and the network of their associated researcher and institutions, 
the following strategic tools have been planned and implemented: 
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e Formation of a Scientific Committee with participants from each of the 
nodes to keep track of the activities related to the repository (request of 
access, both individual and from agencies) and to set up rules for the 
use of the video material also in future. 

e Signing of an agreement among the groups that made the project in 
order to ensure continuity to the research and development work. 

e Elaboration of a Costs Plan, to maintain the repository active and to 
develop it. 

e Contact educational agencies in each country to involve them in the 
project. 

e Contact private operators to involve them in the project (mainly 
publishing houses). 


5. Benefits of exploitation 


To ensure that the knowledge created within the FAMT&L project will 
also be used in future is one of the aims of the exploitation. The expected 
benefits of the exploitation can be various: 


5.1. Scientific exploitation 


The project’s results have been made known to scholars who do research 
in the field, the collected information and database of the project results 
are being disseminated in conferences, workshops, panel discussions and 
scientific publications. Furthermore, the results have been integrated into 
courses taught at universities. 


Dissemination of innovative educational approaches 

The FAMT&L project’s results in their entirety have been and still 
will be published and disseminated by the FAMT&L website, newspaper 
articles, scientific journals, focused events, workshops and conferences, in 
particular by publicizing the platform with video materials and innovative 
models of teacher training. 


5.2. Planning and outcomes 


The following table gives an overview of the project planning and its 
outcomes: 
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Tab. 2 - Exploitation outcomes 


Outcome 


Target group 


Objectives 


Indicators 
for measurement 
of success 


Exploitation Plan 
(Planning Document) 


Partners 


To keep track of 
the state of the art 
of our work and to 
exchange info 


At least one person 
for each node has 
been involved in the 
on-going draft of the 
document 


Agreement of 
co-operation with 
schools 


Partners and 
stakeholders 


Have a standard 
frame of work for 
cooperation with 
schools for all our 
nodes 


Number of schools 
for the associated 
partners: 

5 Italy, 2 Cyprus, 

1 France, 2 
Switzerland, 2 
Netherlands 


Short videos 
(Videos) 


Partners and 
stakeholders 


Use as main tool for 
the elaboration of 
training stra-tegies 
for FA. 

Gathered in our 
repository 


Number of videos: 

126. 

UNIBO (90) - UCY 

(16) - SUPSI (11) — 

UCP (9) 

Many students and 
teachers involved. 


Briefings with policy 
and decision makers, 
educational agency 
and operators 

at national level 
(exploitation events) 


Partners and 
stakeholders 


Spread awareness 
about FA and its 
relevance; improve 
funding possibilities 
and spaces for 
training 


See section 4.2 


Training meetings 
with teachers 


Partners and 
stakeholders 


Pilot courses have 
been the main 

tool to introduce 
teachers to video 
analysis as a mean 
to improve their use 
of FA techniques 


Pilot courses for 
teachers have been 
hold by the members 
of the project with 
different formats 

(6 to 30 hours 
modules, more 

than 70 teachers 
involved). They have 
been successful, 

as proven by the 
results of the relative 
questionnaires 


Seminars about FA 
and the project 


Partners and 
stakeholders 


Introduce teachers 
and university 
students to FA and 
results of the 


Several seminars 
with good number 
of participants 
(university 


project researchers and 
students, in service 
and pre-service 
teachers) 
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6. Sustainability 


It is part of the ambition of the FAMT&L project to make a stable 
contribution to the development of strategies that enhance the use, awareness 
and development of FA in the teaching of mathematics. 

One of the main goals of the project is keeping and enlarging our web 
repository that should be one of the reference points in Europe for the use 
of videos in FA training. The project members should also aim at creating 
a sustainable network and at writing state of the art reports, to facilitate the 
circulation of information. 

Other activities can be implemented on an “as needed” basis, but at least 
the basic function as “observer” and “knowledge hub”, should be safeguarded 
and made sustainable also after M36 of the project. 

Unibo is working to make the sustainability of the repository possible; 
what is mainly needed is to have a good maintenance of the repository, to 
elaborate good rules and means for the access to the materials, to allow 
development processes. 

For these objectives, the FAMT&L project’s staff has elaborated an 
agreement for the adequate use of the repository in the five countries 
which has been undersigned by the project’s groups; such an agreement 
will provide a constant scientific supervision of this tool and a coherent use 
of it for the formation of teachers about formative assessment. 

At present, the repository is intensively used for training courses for 
in-service teachers and for university courses for future teachers. Actually, 
University students in Mathematics and Education, as future teachers, are a 
particularly relevant target group for our action. 

The other main objective of the project was to keep a research network 
active and developing, via contacts, exchanges and agreements with several 
actors. 

The following subjects have been involved: 

e the research group in “didattica della matematica” at the University of 

Turin; 

e the international web WiTEC; 

e INVALSI (National Institute for the Educational Evaluation of Instruction 
and Training); 

e University of South Australia (research group in the teaching of 

Mathematics); 

e an international research group named Video 4 teachers, which, also 
following our work, has been formed at the University of Cagliari (see 
http://sites.unica.it/video4teachers). 
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OPEN ACCESS 


la soluzione FrancoAngeli 


Il presente volume é pubblicato in open access, ossia il file dell’intero lavoro é 
liberamente scaricabile dalla piattaforma FrancoAngeli Open Access 
(http://bit.ly/francoangeli-oa). 


FrancoAngeli Open Access è la piattaforma per pubblicare articoli e mono- 

grafie, rispettando gli standard etici e qualitativi e la messa a disposizione dei 
contenuti ad accesso aperto. Oltre a garantire il deposito nei maggiori archivi 
e repository internazionali OA, la sua integrazione con tutto il ricco catalogo 

di riviste e collane FrancoAngeli massimizza la visibilità, favorisce facilità di 
ricerca per l’utente e possibilità di impatto per l’autore. 


Per saperne di più: 
http://www. francoangeli.it/come pubblicare/pubblicare 19.asp 


I lettori che desiderano informarsi sui libri e le riviste da noi pubblicati 
possono consultare il nostro sito Internet: www.francoangeli.it e iscriversi nella home page 
al servizio “Informatemi” per ricevere via e-mail le segnalazioni delle novità. 
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Ricerca-Formazione 
diretta da D. Capperucci, R. Cardarello, B. Losito, I. Vannini 


Ultimi volumi pubblicati: 


PATRIZIA MAGNOLER, 1l tutor. Funzione, attività e competenze (disponibile anche in e- 
book). 
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UESTO LIBRO TI È PIACIUTO? 


Comunicaci il tuo giudizio su: 
www.francoangeli.it/latuaopinione.asp 


VUOI RICEVERE GLI AGGIORNAMENTI 
SULLE NOSTRE NOVITÀ 
NELLE AREE CHE TI INTERESSANO? 


à FrancoAngeli 
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Bi aM, 4 Sottoscrivi en Iscriviti 
Seguici in rete (9 i nostri feed RSS | alle nostre newsletter 


RICERCA - FORMAZIONE 
FORMATIVE ASSESSMENT FOR MATHEMATICS TEACHING AND LEARNING 


This book aims at describing pathways and achievements of the LLP Comenius Project “FAMT&L — 
Formative Assessment in mathematics k teaching and learning”. The main purpose of this research 
project was to encourage the use of formative assessment in the teaching-learning process carried out by 
mathematics teachers in lower secondary school. The five partner universities involved in the Project have 
been the Alma Mater Studiorum University of Bologna (Italy), the University of Applied Sciences and Arts 
(Southern Switzerland), the University of Cergy-Pontoise (France), the University of Cyprus, and the 
Hogeschool Inholland (Netherlands). 

Through the voices of the several actors involved in this project, the book wants to contribute to the 
international debate on issues regarding teachers’ professionalism in the European school systems: 
specifically, on the development of the rtofestona ll of teachers in the field of assessment practices 
and on the use of formative assessment in the classroom. 


Federica Ferretti, PhD in Mathematics Education, is Researcher in Mathematics Education at the 
Free University of Bolzano-Bozen, Faculty of Education. Her main research interests concern the 
Didactic Contract at all school levels, formative assessment in mathematics and formative use of 
standardized assessments. For years she has been involved in mathematics teachers professional 
development. She is member af CRESPI Center. 


Paraskevi Michael-Chrysanthou is a member of the Special Teaching Staff of the Department 
of Education at the University of Cyprus (2011-2017). She was a Research Associate in the program 
of the Cyprus Research Promotion Foundation “The role of representation in learning functions and 
geometry in the context of the shift from lower secondary education (gymnasium) to upper 
secondary education (lyceum)” and in the Research Program of the University of Cyprus “Spatial 
Ability and Geometrical Thinking Development”. She also participated in the project “History of 
Mathematics Education in Cyprus 1878-1960” of the Centre of Scientific el of the Ministry of 
Education and Culiure. 


Ira Vannini, Associate professor of Educational Research at the Department of Education 
"Giovanni Maria Bertin" of the Alma Mater Studiorum University of Bolina: Her research activities 
(also in collaboration with CRESPI-Center of Research on teacher professionalism) are focused on 
topics such as formative evaluation research and curricular planning; studies on teacher training with 
the use of technics of video-analysis in classroom; use of formative assessment and classroom 
observation. 
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